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Background 


In September 2000, Canada’s federal and provincial 
ministers of health agreed to develop an Electronic 
Health Record (EHR)—defined by the Ministry of 
Health and Long-Term Care (Ministry) as “a secure 
and private lifetime record of an individual’s health 
and care history, available electronically to author- 
ized health providers.” All 10 provinces and three 
territories have been working with the assistance 
of the federal government to develop EHR systems 
that, it is hoped, will eventually interconnect, to 
allow, for instance, a physician in Toronto who is 
treating a patient to instantly access that patient’s 
health records, whether that patient lives just down 
the street or in another province. 

One Canadian study estimates it will cost 
about $10 billion to fully develop an EHR for all 
Canadians but that such a system could save the 
country $6 billion in health-care costs each year. 
Canadian Health Infoway (Infoway), an organiza- 
tion created by the federal government in 2001 
to help the provinces develop EHRs, concurs with 
these estimates. The Infoway’s stated goal is that 
50% of Canadians will have an EHR by 2010 and 
100% by 2016. Although it is funded by the federal 
government, the Infoway reports to all federal, 
provincial, and territorial deputy ministers of 
health. 


Mts Ontario's Electronic 
Health Records Initiative 


The Infoway estimates that there are about 
2,000 health-care “transactions” in Canada every 
minute, or more than 1 billion transactions each 
year, including: 


440 million laboratory tests; 

e 382 million drug prescriptions; 

e@ 332 million visits to physicians’ offices; 
e@ 35 million diagnostic images; and 

e 2.8 million in-patient hospitalizations. 

The vast majority of these transactions are 
recorded on paper or other media, such as x-ray 
film, and then physically stored. The Ministry 
envisions a system where all of these records could 
eventually be collected, stored, and retrieved in 
digital form, and then made available on-line to 
authorized health-care providers, with all Ontario 
citizens having access to their own EHR. 

In order to work as envisioned, an EHR system 
requires four fundamental components: 

1. asecure network on which patient data can 
travel; 

2. applications that enable users to record, store, 
and retrieve that patient data; 

3. patient data, such as treatment history, test 
results, diagnostic images, and prescribed 
medications, in digital form; and 

4. terminals or access points from which users 
can input and retrieve patient data. 

In Ontario, planning for EHRs began in the late 
1990s. In 2002, the Ministry created the Smart 
Systems for Health Agency (SSHA). SSHA was to 
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oversee the creation of a secure electronic network 
and the connecting of the medical community to 
this network, while the Ministry was to be respon- 
sible for the overall EHR strategy and the develop- 
ment of the clinical applications and associated 
databases that would run on the network. The 
province has spent more than $1 billion since 2002 
on EHR-related activities, with SSHA accounting 
for about $800 million of these expenditures. 

In September 2008, the Ministry created the 
eHealth Ontario agency to take over both SSHA and 
the Ministry’s own EHR initiatives. The agency’s 
mandate is to have an Electronic Health Record in 
place for Ontarians by 2015. 

Shortly after it began operations, the eHealth 
Ontario agency found itself embroiled in con- 
troversy when it was widely reported that it had 
handed out millions of dollars in untendered 
contracts to consultants. This controversy led to the 
resignation of the eHealth Ontario agency's CEO 
and the Chair of the board of directors. 

A more detailed chronology of events relating 
to the province’s EHR initiative is provided in the 
Appendix. 


REQUEST BY THE MINISTER AND THE 
STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 


In response to questions in the Legislature in April 
and May 2009 regarding procurement issues at 
the eHealth Ontario agency, both the Premier 
and the Minister of Health and Long-Term Care 
advised the House that one of the audits already 
under way by my Office related to eHealth. On 
June 2, 2009, the Minister called me to ask when 
the results of this audit could be made public. 
Ladvised the Minister that under section 12 of 
the Auditor General Act, I was required to report the 
results of all audit work for the year in my Annual 
Report, which is normally tabled in early December. 
However, I also advised the Minister that if he was 
to make a formal request under section 17 of the 
Act, then I would be able to issue this audit as a 





special report when it was completed and make it 
public. Later that same day I received a letter from 
the Minister formally requesting under section 17 of 
the Act that I issue this audit as a special report as 
soon as it was complete and make it publicly avail- 
able at that time. 

On June 3, 2009, the day following the formal 
request from the Minister, the Standing Committee 
on Public Accounts met and passed the following 
motion by unanimous vote: 


That following the Auditor General’s 
completion of his value-for-money audit 
of eHealth Ontario, the Standing Commit- 
tee on Public Accounts of the Legislative 
Assembly of Ontario calls on the Auditor 
General to release that chapter of his 
Annual Report in a Special Report to the 
Speaker; and that prior to the tabling 

of this report, the Auditor General may 
inform the Deputy Minister of Health and 
Long-Term Care of his opinions, observa- 
tions, or recommendations. 


Audit Objective and Scope 


The objective of our audit, which was established 
months before the controversy arose, was to assess 
whether the Ministry of Health and Long-Term Care 
(Ministry) and the new eHealth Ontario agency 
had adequate systems and procedures in place to 
help ensure that an Electronic Health Record (EHR) 
for Ontarians was being implemented in a cost- 
effective manner, and to assess progress in meeting 
the government’s commitment to develop an EHR. 

In conducting our audit work, we consulted 
with the Office of the Auditor General of Canada 
and five other provincial legislative audit offices 
that were also examining EHR-related activities in 
their respective jurisdictions. Each of the provincial 
audit offices planned to table their own EHR report 
in their provinces. The federal Auditor General 


anticipates tabling her report, which may include 
observations from the various provincial reports, in 
Parliament in spring 2010. 

We did not examine the privacy aspects of 
Ontario’s EHR initiative. The Office of the Informa- 
tion and Privacy Commissioner conducted a com- 
prehensive privacy review of SSHA in 2007 based 
on the Personal Health Information and Privacy Act, 
2004. We hired a privacy consultant recommended 
by the Commissioner to help us determine what 
privacy-assessment work we should consider, given 
the review that had already been undertaken. 

The consultant observed that the eHealth Ontario 
agency’s work on privacy was still at a preliminary 
stage and concluded that it would not be useful to 
review it at this time. We accepted this recommen- 
dation and, accordingly, excluded privacy from the 
scope of our audit. 

The documents we examined included applic- 
able legislation and regulations, requests for 
proposals, submitted proposals from vendors, 
proposal evaluations, contracts, invoices, payment 
records, expenditure reports, board minutes, pro- 
ject charters, progress reports, network utilization 
data, service agreements, correspondence, and 
agreements respecting the relationship between 
the Ministry and both SSHA and the eHealth 
Ontario agency. In addition, we met with many 
EHR stakeholders, both inside and outside the 
government, including members of the former 
senior management of SSHA, the management of 
the eHealth Ontario agency, the eHealth Ontario 
agency’s board of directors, the former CEO of 
the eHealth Ontario agency, and the former Chair 
of the eHealth Ontario agency. We also talked to 
representatives of some of the consulting firms 
involved, who willingly provided us with certain 
additional documentation. 

It should be noted that our observations about 
procurement practices involving private-sector con- 
sultants apply only to the actions of public servants. 
They are in no way intended to pertain to or reflect 
on any practices that consultants followed or their 


performance. 
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Our audit followed the professional standards of 
the Canadian Institute of Chartered Accountants for 
assessing value for money and compliance. We set 
an objective for what we wanted to achieve in the 
audit and developed audit criteria that covered the 
key systems, policies, and procedures that should 
be in place and operating effectively. These criteria 
were discussed with senior management. We then 
designed and conducted tests and procedures to 
address our audit objective and criteria. 

We noted that while Internal Audit completed 
due-diligence reviews of two of the larger EHR 
projects at the planning stage before they were 
approved to proceed, it conducted no other audit 
work of significance at the Ministry regarding 
its eHealth activities, at SSHA, or at the eHealth 
Ontario agency. Accordingly, we did not rely on the 
work of Internal Audit in conducting our work. 


DELAY IN STARTING OUR AUDIT 


We typically complete several value-for-money 
audits annually at the Ministry of Health and 
Long-Term Care, and normally we receive the full 
co-operation of Ministry staff. Unfortunately, this 
was not the case for this audit. 

I first wrote to the Deputy Minister in the late 
summer of 2008, advising him of this audit. I stated 
that the EHR was to be the subject of the audit. We 
had planned on starting fieldwork in the fall. As 
our planning proceeded, we requested access to 
the Ministry’s eHealth Program Branch office and 
working accommodations for our field auditors, 
as is our normal practice. Despite repeated efforts 
over the course of several months, we were granted 
neither access nor accommodations until early 
February 2009. 

Initially, the reason cited for the lack of access 
was space limitations, so my staff conducted 
research and received information relating to our 
audit from the Ministry while waiting for an accept- 
able working area to be freed up for their use. 
However, once the space was finally made available 
in late December, we were surprised to learn that 
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access was still being denied us, this time on the 
grounds that the Ministry had not yet agreed to the 
scope of our audit or to the overall audit objective 
of assessing progress on its EHR initiative. Senior 
management informed us that under the new 
eHealth Ontario agency strategy, in development 
at the time, the government planned to focus on 
“quick-win” projects—in diabetes management, 
medication management, and wait times—that had 
more immediate clinical value. We were told that, 
since the Ontario government “does not have an 
EHR strategy,” but rather is working on a broader 
eHealth strategy, its progress on the EHR should 
not be our focus, and, until this was resolved, we 
did not have the Ministry’s approval to commence 
our audit fieldwork. We were granted access only 
after I intervened directly with the Deputy Minister. 





The focus of members of the Legislature and the 


media with respect to Ontario’s Electronic Health 
Record (EHR) initiative over the last few months 
has primarily been on the awarding of contracts 

at the eHealth Ontario agency without an open 
competitive process. In essence, allegations were 
made that contracts were sole-sourced to a number 
of consulting firms or individual consultants by the 
CEO of the eHealth Ontario agency or her appoin- 
tees. Our work indicated that this was undoubtedly 
the case. But we also noted questionable procure- 
ment practices at the Ministry of Health and 
Long-Term Care’s (Ministry's) eHealth Program 
Branch and at the Smart Systems for Health Agency 
(SSHA), albeit to a lesser extent. However, the 
main focus of our work as originally planned was 
not on consultant procurement but rather on the 
overall progress of the EHR initiative given the 

$1 billion in EHR expenditures made over the past 
seven years. In this summary, we first discuss our 
observations on that issue, then our observations 
on procurement. 
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It is important to remember what the “e” in 
eHealth stands for—“electronic.” In other words, 
Ontario’s EHR initiative is essentially a number of 
large, complex, and interrelated information tech- 
nology (IT) projects. The most important standard 
for judging the ultimate success of any IT project is 
not whether it was delivered on time or on budget, 
or what technology was used, but rather whether 
the system meets the needs of its users. Although 
some applications, such as Telemedicine, have been 
developed, there is no doubt that Ontario does 
not yet have an eHealth system that is meeting the 
needs of medical practitioners or the public. In 
terms of EHR progress, the Canada Health Infoway 
(Infoway) has reported that Ontario is near the 
back of the pack compared to most other provinces. 

About $800 million of the total $1 billion that 
has been invested in the EHR initiative was incurred 
by SSHA, primarily in building and operating a pri- 
vate IT network and connecting the medical com- 
munity to it. The value of this investment, at least 
to date, has not been realized. Part of the problem 
lies in operational issues, and we noted areas 
where costs could have been reduced. But our main 
concern is that the network remains significantly 
underutilized because as yet there is insufficient 
health-related information on it. Although some of 
the broader EHR applications were begun earlier, 
many of the critical EHR application projects were 
not begun until 2008. Responsibility for this rested 
with the Ministry. 

We grouped our observations into five main 
themes: strategic planning, oversight, progress on 
EHR projects, the use and procurement of consult- 
ants, and expenditure management. 


STRATEGIC PLANNING 


Upfront strategic planning, essential to a major 
initiative like the EHR, had not been properly com- 
pleted until years after the government launched 
the initiative in the early 2000s. What such a stra- 
tegic plan should have clearly defined were: 

e the specific needs of users; 


how these needs would be met; 
who would be accountable for delivery; 
what the timelines would be; and 


how much each component of the system was 
expected to cost. 

It was only in March of 2009 that the first-ever 
eHealth strategic plan was finalized and approved. 
Although we welcome this plan, it is an overall 
eHealth strategic plan, not an EHR plan. We believe 
it needs to be supplemented with a plan that more 
specifically addresses the 2015 EHR target. 

SSHA’s job—to use a highway transporta- 
tion analogy—was to build and operate a well- 
maintained, secure “super-highway” to which 
medical practitioners would be connected and on 
which the EHR and other eHealth applications 
would run. However, partly because of short- 
comings in upfront strategic planning, the clinical 
information that medical practitioners need for 
decision-making is not available. The underlying 
applications with the clinical information (that 
is, the vehicles and their loads) simply are not 
there. With few vehicles capable of travelling on 
the highway and few goods to deliver, it is not 
surprising that, notwithstanding the approximately 
$72 million in direct operating costs annually being 
spent to keep the highway up and running and the 
$800 million incurred by SSHA to date, there has 
been little traffic on it. It appeared to us that the 
Ministry did not anticipate the risk of having the IT 
infrastructure operational but having no applica- 
tions available for some time—a situation that 
may have been avoided had there been an overall 
strategic plan driving the EHR initiative from the 
very beginning. 

In addition, the fact that SSHA was participating 
in an initiative whose endpoint was to deliver an 
Electronic Health Record to all Ontarians was not 
satisfactorily addressed. SSHA developed its private 
network for use only by the health-care community; 
it will need to integrate securely and effectively 
with other networks such as the Internet in order to 
link all Ontarians as users. 
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OVERSIGHT 


Throughout the years, oversight of the EHR initia- 
tive has not been effective. This was reflected in 
the relationship between the Ministry and SSHA, 
which was unproductive. This was also reflected in 
governance problems at the new eHealth Ontario 
agency that enabled the CEO to wield considerable 
power. 

The decision to have two entities at arm’s length 
from each other deliver the EHR solution—SSHA 
responsible for the underlying infrastructure and 
the Ministry’s eHealth Program Branch responsible 
for the overall EHR strategy and the related applica- 
tions—was, in our opinion, inherently problematic. 
The success of this plan depended on both parties 
having a cohesive and co-operative working rela- 
tionship. This was never the case. There was little 
co-ordination or co-operation between SSHA and 
the Ministry’s eHealth Program Branch. We were 
informed by numerous parties on both sides that 
the relationship was marred by mutual mistrust 
and confusion over roles and responsibilities. The 
dysfunctional relationship between the two entities 
and the evident discontent of the medical commun- 
ity should have “raised the red flag” sooner that 
something needed to be done. 

SSHA has been blamed for spending more than 
$800 million and getting little in return. In essence, 
SSHA has been made the scapegoat for the lack 
of progress to date on the whole EHR agenda. In 
our opinion, this is somewhat unfair. SSHA must 
certainly take its share of the responsibility for 
building a system that has consistently had oper- 
ational problems and could have been run more 
cost-effectively. And while SSHA did begin to exam- 
ine the potential of advances such as more secure 
Internet technologies that could contribute to fixing 
the problems, these have yet to be implemented. 
Deloitte Consulting highlighted user dissatisfac- 
tion with the network in its 2006 review of SSHA, 
reporting that there had been an “erosion of confi- 
dence” in the user community. However, SSHA did 
develop and deliver a secure electronic network, 
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and we suspect that users would have been much 
more willing to put up with network operational 
issues if the network was providing them with more 
useful clinical information. 

Although we support the eventual decision to 
make the delivery of the EHR system the respon- 
sibility of just one entity, we believe that more 
rigorous oversight of the eHealth agenda would 
have resulted in this decision being made much 
earlier. Two factors contributed to such oversight 
not being sufficiently rigorous—SSHA’s status as 
a Crown agency, operating in a more arm’s-length 
capacity; and the lack of clearly defined eHealth 
deliverables linked to an approved strategic plan. 
We also noted that the decision to give responsibil- 
ity for the EHR to a new agency in the absence of 
effective mechanisms for monitoring the agency’s 
operations empowered the agency’s CEO and her 
appointed team, allowing them to bypass standard 
procurement practices and make the decisions and 
award the contracts that so captured public atten- 
tion. We present examples of these issues in the The 
Procurement and Use of Consultants section of this 
Summary. 


PROGRESS ON EHR PROJECTS 


Progress on EHR projects has been slow, and details 
of implementation delays are provided on a project- 
by-project basis later in the report. Two overall 
observations were: 
e Although by March 31, 2007, more than 
$400 million had been spent building the 
EHR network and connecting the health-care 
community to it, investments in the EHR 
applications that would run on the network 
amounted to only about $100 million. As well, 
the medical community can use few of these 
applications as yet because most have yet to 
be completed. Only in the last year, with the 
completion of the eHealth Strategic Plan and 
the approval of the funds necessary to execute 
it, have investments in EHR applications 
approached the level of investment in the net- 


work itself. Work on the applications should 
have proceeded more in tandem with work 
on the network, but the timing of application 
development was out of sync with the spend- 
ing on the IT infrastructure. 

From October 2008 through June 2009—the 
transition period when the eHealth Ontario 
agency was established—many EHR projects 
were put on hold awaiting clarification as to 
their scope and direction. Revised project 
charters for all EHR projects were not com- 
pleted until July 2009. This contributed to the 
pushing back of a number of expected comple- 
tion dates for various projects. 

It should be noted that the recent replace- 
ments of eHealth Ontario’s board Chair and 
CEO mark the fourth such overhaul of leader- 
ship at eHealth Ontario and its predecessor, 
SSHA. Each of these overhauls brought with 
it its own period of transition where progress 
on the initiative’s objectives was slowed or, at 
times, halted. Such slowdowns are somewhat 
to be expected, as a new management team 
naturally must assess the organization it has 
inherited before restructuring it ina manner 
that will mitigate the problems and errors that 
caused the management turnover in the first 
place. Nevertheless, with each such change 
there is a price to be paid in terms of lost time, 
little perceived benefit for the costs incurred 
during the transition period, weakened 
employee morale, and loss of stakeholder 
confidence in the organization. 

EHR projects have for the most part not met 
expectations. We questioned the depth of the 
upfront planning before EHR projects were 
initiated. Such planning should have included 
identifying all of the resources—staff hours, 
consultant hours, and other costs such as 
equipment and software—needed to complete 
the projects and a strategy for procuring them 
in the most economic and efficient manner. 
Instead, resource needs were all too often 
identified and filled on an ad hoc basis as 


projects progressed from one phase to another. 
Further, these resource needs were usually met 
via costly short-term consulting engagements 
that often were continuously renewed. In addi- 
tion, project expenditures and deliverables 
were tracked separately without an adequate 
oversight mechanism to integrate this informa- 
tion and thus ensure that deliverables were 
completed as the related costs were incurred. 
We also noted projects with significant “scope 
creep,” projects with unrealistic timelines that 
could not be met, projects where the require- 
ment for integration with other components 

of the EHR was not adequately considered, 
and perhaps most importantly, projects 

where the end product is not yet meeting user 
requirements. 

@ The network built by SSHA is not being man- 
aged cost-effectively. For instance, the agency 
is paying $2.5 million per month to maintain 
circuits that are either inactive, almost 
inactive, or significantly underutilized. 


THE USE AND PROCUREMENT OF 
CONSULTANTS 


Extent of Reliance on Consultants 


Although we acknowledge that the highly tech- 
nical nature of the EHR initiative necessitates 
some degree of specialist consulting advice, the 
fact that the development of an EHR had been on 
the government’s agenda as far back as the early 
2000s caused us to question the heavy, and in 
some cases almost total, reliance on consultants. 
This reliance continued to increase over time. This 
was particularly the case at the Ministry, which in 
2007 consolidated all of its eHealth projects into an 
eHealth Program Branch. By 2008, the Branch was 
engaging more than 300 consultants compared to 
fewer than 30 full-time ministry employees—even 
4 number of senior management positions were 
held by consultants. Consultants were not only 
managing other consultants but also at times had 
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the authority to hire more consultants, sometimes 
from their own firm. Numerous consultants had 
been retained for a number of years to work on a 
variety of different assignments. For instance, one 
consultant had been employed for seven years and 
his firm had been paid $2 million; another had 
been employed for six years and the firm had been 
paid $2.4 million, and $6.9 million had been paid 
with respect to six other consultants who have been 
engaged continuously on EHR projects since 2005. 


Procurement 


As noted at the beginning of this Summary, it was 
the procurement practices of the eHealth Ontario 
agency that first captured the attention of the 
media and the Legislature. However, we found 
that procurement problems were not exclusive to 
eHealth Ontario; the Ministry’s eHealth Program 
Branch and SSHA also engaged in certain question- 
able procurement practices. 


Procurement at eHealth Ontario 
In our opinion, the allegations that contracts were 
awarded to certain consultants and vendors with- 
out giving other firms the chance to compete for the 
business are largely true. In fact, we estimate that 
two-thirds of the value of all eHealth Ontario con- 
tracts was sole-sourced. Allegations that the agency 
showed favouritism in awarding some of these 
contracts are also true. In addition, we were aware 
of the allegations that “party politics” may have 
entered into the awarding of contracts and that 
those awarding the contracts may have obtained a 
personal benefit from the firms getting the work— 
but we saw no evidence of this during our work. 
The CEO of the eHealth Ontario agency was 
appointed in October 2008 and reportedly told 
that there were serious problems with the eHealth 
agenda and that overcoming these problems 
required urgent action. She wanted to build her 
own team—people she had worked with in the past 
or whom she personally recruited—rather than 
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rely on ministry or SSHA staff or their consult- 
ants. The CEO felt she had the implied, if not the 
formal, authority to do whatever was necessary to 
get the job done. If this meant personally selecting 
the firms and individual consultants that she 
wanted, so be it. In our opinion, the CEO’s prior 
relationships with a number of the firms and indi- 
viduals were one of the factors in her hiring and 
procurement decisions, and this does constitute 
favouritism. 

We do not accept “urgency” or the fact that 
the consultants had relevant experience as valid 
reasons for this sole-sourcing of contracts; neither 
do we accept the contention that the procurement 
policies in place, or Treasury Board’s earlier grant- 
ing of a short-term exemption from competitive 
tendering for certain contracts, justified the sole- 
sourcing. 

The eHealth Ontario agency had procurement 
policies that allowed exceptions to competitive 
tendering when, for example, the need for the work 
was so urgent that there was no time to competi- 
tively tender or only one vendor was available for 
the service required. However, the sole-sourced 
contracts we reviewed did not, in our opinion, meet 
any of these exception criteria. For example: 

e Anexternal recruiting firm was hired dir- 
ectly by the CEO on a sole-sourced basis for 
$1 million to help in the hiring for 15 senior 
management positions. In our opinion, there 
was no reason why this contract could not 
have been competitively tendered. As well, 
the eHealth Ontario agency paid most of 
these fees up front. We noted that only five 
of the 15 positions were filled when the 
contract was terminated—yet no money had 
been requested by or returned to the eHealth 
Ontario agency. 

e For one significant contract, there was an 
appearance of a competitive-tendering 
process, when in reality senior management 
already knew to whom it wanted to award 
the contract. After receiving responses to an 
invitation to tender for the project, a firm was 


selected as the successful bidder even though 
its bid price of $3.1 million was more than five 
times higher than another qualified bidder’s 
price of $570,000 and well above the project’s 
approved budget of $700,000. Only this high- 
est-bidding firm was asked to resubmit its bid 
to bring it in line with the approved budget. 
The firm revised its proposal, reduced its total 
quote to $737,000, reduced the time frame 
over which the work would be performed, and 
significantly reduced the resources it would 
devote to the two main deliverables. 

The firm’s original proposal was to employ 
essentially all of its available staff. Although 
the revised proposal removed three of these 
personnel, at about the same time the firm 
submitted an unsolicited proposal to engage 
these three consultants. The next day, the 
eHealth Ontario agency prepared an excep- 
tion-to-competitive-procurement request, and 
these three consultants were procured for 
$594,000 to conduct the proposed work, with 
the work to commence on the same date as 
the work under the original contract. In our 
opinion, the agency’s intent was obviously to 
award this work to the favoured firm, regard- 
less of the responses received to the request 
for proposals it issued. 

We also considered the extent to which the 
eHealth Ontario agency’s board Chair and board 
members were aware of the procurements that 
management was making. Had they condoned, 
either directly or implicitly, how business with 
the agency’s suppliers was being conducted? We 
reviewed all board minutes and other related docu- 
mentation and had discussions with the board as a 
whole, with the board Chair, and with individual 
board members. 

We believe that the Chair, an individual with 
a credible record in health-care transformation, 
accepted the position of Chair of the eHealth 
Ontario agency in good faith and that his sole inter- 
est in doing so was to help move the EHR agenda 
forward. Although we saw no evidence that the 


board asked for any details on procurements, we 
believe that the board was under the impression, 
based on information provided to it, that consult- 
ants and other vendors were being procured in 
accordance with established policies. 


Procurement at the Ministry’s 

eHealth Program Branch 
The eHealth Ontario agency was not alone in 
using questionable procurement practices. At the 
Ministry’s eHealth Program Branch, we noted the 
following: 

@ Over a four-month period, a consultant hold- 
ing a key management position inside the 
Ministry was involved in awarding five addi- 
tional contracts with a total value of $1.3 mil- 
lion to the consulting firm he was associated 
with. 

e Vendor proposals were not properly evalu- 
ated for a significant number of the contracts 
we sampled, and many were approved 
without sign-off by all appropriate parties. 
For example, a single individual chose which 
vendors to invite and made the sole decision 
on whom to hire for more than 30% of the 
contracts we sampled. We noted contracts 
that were signed on the same date as the 
closing date of the request for service, and 
work performed prior to the contracts being 
awarded. These are indications that, even if a 
competitive process was followed, it was more 
of a formality than a true competition—the 
decision as to which firm should get the work 
had already been made. 

e The Ministry’s procurement policies stipulate 
that contracts above a certain amount should 
be put to tender. We found that contracts were 
often subdivided into multiple contracts for 
smaller amounts, which resulted in the ten- 
dering requirements being bypassed. 

e Ministry requests-for-services proposals were 
often incomplete and unclear as to what 
deliverables were expected. This at times 
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resulted in large variances in the submitted 
proposals’ projected costs. In one noteworthy 
case, the Ministry received two proposals 

in response to a request for services—one 
with estimated costs of $60,000 and the 
second with estimated costs of $600,000. The 
Ministry selected the most expensive vendor 
without any documented explanation. In 
other cases, even for large contracts that, with 
extensions, accumulated to over $1 million, 
the deliverables were not clearly specified and 
it was often difficult to determine what work 
had been performed with respect to those 
contracts. 

@ The Ministry allowed one consulting firm, 
which was initially awarded work on several 
projects through an exemption from competi- 
tive procurement, to obtain significant follow- 
up work, largely owing to its entrenched 
position. For one project, it was awarded 10 
related contracts for a value totalling $4.1 mil- 
lion over a period of 24 months and eight 
related contracts associated with another 
project for a total of $2.9 million over a period 
of eight months. 

To sum up, too many procurements at the 

eHealth Ontario agency and, to a lesser extent, 

at the Ministry’s eHealth Program Branch and at 
SSHA were the product of rushed decision-making; 
the acceptance of expediency over thoroughness; 
the routine defence that the work was of an emer- 
gency nature and therefore justified the bypassing 
of normal procurement controls; procedural 
shortcuts; poor, absent, or contradictory documen- 
tation; and, of particular concern, the concentra- 
tion of decision-making power in the hands of a 
few individuals with no compensating controls to 
ensure their decisions were appropriate. Sound and 
reasonable policies were in place to ensure that all 
suppliers could fairly compete for government busi- 
ness and that tax dollars would be prudently spent, 
but all too often the rules were not followed. We 
have come across this in other audits—ineffective 
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oversight and broken rules go together like a horse 
and carriage. 


EXPENDITURE MANAGEMENT 


At both agencies and the Ministry’s eHealth 
Program Branch, we noted that policies and pro- 
cedures for approving and monitoring payments to 
consultants were inadequate. For instance, consult- 
ant invoices were routinely paid on the basis of the 
number of hours billed. Although we were advised 
that progress on work was monitored in informal 
ways, there was little evidence—or documentation 
to show—that payments had been related to the 
completion of project deliverables. 


RECENT ACTIONS TAKEN BY EHEALTH 
ONTARIO AND THE MINISTRY 


Management at the eHealth Ontario agency has 
informed us that it has taken a number of actions 
since the completion of our fieldwork to improve 
operations. Among them are the following: 

e New policies on procurement and delegation 
of authority have been approved. 

e The procurement function has been strength- 
ened—all areas must now develop plans 
identifying anticipated annual procurements, 
and quarterly procurement reports will be 
provided to the board. 

e Fee-for-service consultants will be eliminated 
in all areas other than program development 
and implementation, with the number of con- 
sultants being reduced from more than 330 to 
less than 170 by the end of the current fiscal 
year. 

e An eHealth Privacy Strategy with targets for 
the 2009/10 fiscal year has been developed. 

e A network service cost-reduction strategy has 
been put in place, with savings of $1.2 million 
annually secured through negotiations with 
one of the agency’s major network service 
providers. 


The Ministry informed us that it also has 
recently introduced measures to improve its pro- 
curement controls: 

@ It has created a Financial Management Branch 
to oversee its workforce, including consult- 
ants, and to centralize all transfer payments. 

e It recently restructured its Supply and Finan- 
cial Services Branch to enhance oversight cap- 
acity for all procurement within the Ministry 
and to centralize its procurement process. 

e An annual business plan that aligns with 
the eHealth Ontario Strategic Plan will be 
developed. 

e It has initiated discussions to improve the gov- 
ernment-wide accounting system to facilitate 
enhanced oversight of contract and payment 
processes. 

These recent actions will help address many of 

the concerns that we have identified throughout 
our report. 


RECOMMENDATIONS 


We took these recent actions into consideration in 
making our recommendations. Other provinces 
and many other countries around the world have 
recognized the benefits of EHRs—both from a 
health-care and a fiscal perspective. Accordingly, 
our recommendations, interspersed throughout 
the report, focus primarily on what we believe to 
be the most important steps that eHealth Ontario 
and the Ministry must take if they are to meet their 
goal of having a fully functional EHR for all Ontar- 


ians in place by 2015. 





The Ministry of Health and Long-Term Care 
(Ministry) and eHealth Ontario welcome 

the report of the Auditor General and his 
recommendations on how to further improve 
the delivery of Ontario’s Electronic Health 
Record (EHR). 


The Ontario government’s eHealth agenda is 
broader than the EHR and focuses on dramatic- 
ally improving patient care through the delivery 
of innovative health information systems. 
eHealth applications are vital tools that will 
support improved access to quality health care 
and health-care innovation and include public 
health surveillance systems, electronic medical 
records in physicians’ offices, diagnostic medical 
imaging, and telemedicine services. Significant 
progress has been made in developing each of 
these systems. 

The government continues to be committed 
to the eHealth agenda and eHealth Ontario’s 
role in establishing EHRs for diabetes patients 
and, in future, for patients with other chronic 
diseases, leading to an EHR for all Ontarians. 

The government is committed to ensur- 
ing that the money spent on eHealth is for 
initiatives that will strengthen and modernize 
Ontario’s health-care system. 

The Ministry and eHealth Ontario have 
introduced new policies and procedures to 
address each of the recommendations contained 
in the Auditor General’s report. 

The government issued a new Procurement 
Directive in July 2009 that requires a competi- 
tive procurement process be used for the acqui- 
sition of all consulting services and prohibits 
consultants from being paid for hospitality, 
food, and incidental expenses. 

In September 2009, the government issued 
a new Travel and Expenses Directive that sim- 
plifies the rules for these expenses; mandates 
on-line training on expense claims for Ontario 
Public Service staff and employees in large 
agencies; and requires that the expenses of 
Ontario Public Service senior managers, cabinet 
ministers, political staff, and agency senior exec- 
utives be posted on-line starting in April 2010. 
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Detailed Audit Observations 





Health-care experts say Electronic Health Record 
(EHR) systems have the potential to save lives, 
reduce health-care costs, and improve patient 
service. Some of the cited advantages of EHRs over 
traditional paper-based record systems are: 

e improved legibility, availability, and retriev- 
ability of data; 

e reduced likelihood of duplicate or unneces- 
sary medical tests; 

e shorter patient waiting lists; 

e more effective physician diagnosis and 
treatment; 

e reduced incidence of prescription errors, mul- 
tiple prescriptions, and inappropriate drug 
combinations that can cause adverse drug 
reactions; and 

e more efficient delivery of health services. 

There are four essential components to an EHR 
system: the network, the applications, the data, and 
the terminals. 

The following outline explains how these four 
components interact, using the analogy of highway 
transportation: 

e@ The network—The network consists of the 
routes and connections via which EHRs are 
to be shared. A developer of an EHR system 
could build its own private network through 
which to channel EHRs or it could make use 
of pre-existing networks built by others. An 
example of the former is the SSHA-built net- 
work, a high-capacity government-controlled 
secure network that is perhaps best akin to an 
Ontario 400 series highway. The best example 
of the latter is the Internet, which is perhaps 
best akin to the vast series of smaller county 
and municipal roads that currently connect 
Ontarians. When it comes to EHRs, choosing 
the latter makes it much easier to connect 
all health-care providers and every Ontar- 
ian to the system. However, the Internet is 
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inherently less secure than the type of private 
network built by SSHA. 

The applications—The applications are the 
instruments through which EHR data are to 
be shared on the network. They are akin to the 
various transport vehicles that travel our high- 
ways daily, delivering the vast variety of goods 
Ontarians need. Different types of vehicles 
deliver different types of goods; likewise, 
different applications will allow the different 
parts of the EHR to be shared. The application 
that allows diagnostic images such as MRI 
scans to be shared will not be the same as the 
application that allows prescription medica- 
tion data to be shared—just as a milk truck is 
different from a cement mixer. But they can 
both use the same highway. 

The data—The data consist of the information 
that people want to share and are akin to the 
goods that are delivered on our highways. 
Data could consist of, for example, a patient’s 
lab test result, a list of medications he or 

she is on, or the patient’s latest x-ray. All of 
these data have to be accurately and reliably 
compiled and available in updatable elec- 
tronic databases so that the applications can 
share the data. Most of the EHR initiative’s 
databases will be separate from each other, 
but when an individual’s EHR appears on the 
computer screen, the applicable data from 

all the different databases will appear in one 
place, as illustrated in Figure 1. 

The terminals—The terminals are the destina- 
tion points being connected via the network. 
In terms of our analogy, the terminals are 
where we want the goods—the data—to be 
delivered. Some of the key terminal points for 
EHRs are the hospitals, clinics, and physician’s 
offices where Ontario’s health-care providers 
do their professional work. One ultimate aim 
of the EHR initiative is to allow Ontarians 
access to their own personal health-care rec- 
ord—every Ontario home that has a computer 


through which residents access the Internet is 
envisioned as an eventual EHR terminal point. 


STRATEGIC PLANNING 


The first, and perhaps most critical, question to 
answer before undertaking any major IT investment 
is, “How will it address a clearly defined business 
need?” This is often spelled out in a business case 
supporting the decision to invest. In the case of 
EHRs, the need for the system appears well estab- 
lished—and this is reflected in the cross-Canada 
governmental support for it (see the Appendix for 
more details). 

One of the most critical issues, and one that 
needs to be resolved early, is how centralized an 
IT system should be. A more centralized system, 
which often focuses more on developing the tech- 
nology capable of serving the widest range of users, 
may well provide more consistent service and data 
access. The risk, however, is that technological 
and structural issues become more important than 
whether the applications are truly meeting user 
needs. A decentralized system makes more allow- 
ance for stand-alone applications customized to 
specific user needs. But there is also a risk that a 
decentralized system would engender information 
silos—subsystems and applications incapable of 
interacting, or being integrated, with related sub- 
systems and applications; subsystems and applica- 
tions may also be duplicated, and it may be difficult 
to identify the problems that need to be solved and 
the priorities that need to be set. 

A key step in addressing user needs and 
developing the appropriate underlying IT infra- 
structure is a well-defined strategy. In arriving at 
this strategy, it is essential that the key stakeholders 
and users of the end product be actively involved up 
front. All too often, those proposing a new system, 
whether they be senior management or individuals 
from the technology side of the business, drive 
the investment agenda without adequate input 
from those who will actually be affected by, or use, 
the new system. Only once these needs are well] 
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Figure 1: Possible Layout of an Electronic Health Record 


Source of figure: the eHealth Ontario agency 
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understood should work proceed. In the case of the 
EHR system, there would be little argument that the 
size, complexity, and diversity of the stakeholder 
and user community made it essential to carefully 
consider in detail the full range of their needs. A 
solid strategy also considers and addresses all rel- 
evant project risks. For instance, do those with the 
assigned responsibility for executing the strategy 
have the necessary depth of experience and project 
management capability? 

A well-articulated strategy also needs to lay out 
a road map for how the various components are 
to be developed, the time frame for completion 
of each component of the system or phase of the 
work, and the estimated costs of each component or 
phase of the system. In the case of an EHR, integra- 
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issues are also key pieces of the puzzle. 

Once completed, the strategy becomes a valu- 
able tool, not just for securing funding and getting 
the project off the ground, but also for evaluating 
progress against the benchmarks contained in it 
throughout the life of the project. This helps ensure 
that major problems, such as costs getting out of 
hand, project deadlines being missed, or products 
not matching up with users’ needs, can be identi- 
fied early and corrective action taken before the 
problems worsen. 

The Ministry did not adequately addressed all 
these key elements of effective strategic planning 
when it launched its EHR initiative or even in the 
subsequent years. Although it did obtain approvals 
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for certain projects that were in support of eHealth 
and an eventual EHR, the Ministry acknowledged 
as late as 2008 that finalizing the eHealth Strategic 
Plan was a work in progress. The Ministry informed 
us that it was given the go-ahead to develop a com- 
prehensive eHealth strategy only when the govern- 
ment announced in 2007 that it would commit to 
an EHR by 2015. And only in 2009 was it provided 
with the significant funding from the 2009 Ontario 
provincial budget to enable it to move ahead. The 
eHealth Strategic Plan was formally approved by 
the government in 2009. 


Consequences of Not Having a 
Strategic Plan—2002 to 2008 


The Ministry had drafted various eHealth strategic 
documents from 2002 onwards, but it did not 
obtain government approval for any of them. 

In fall 2006, Deloitte Consulting completed an 
operational review of SSHA commissioned by the 
Ministry and SSHA’s board of directors because 
of growing concerns about SSHA’s slow progress. 
Among the problems Deloitte identified was the 
absence of a comprehensive government eHealth 
strategy, which resulted in SSHA not being clear 
on its role and not being able to complete its own 
strategy. In 2007, we reviewed SSHA’s efforts to 
address certain key Deloitte recommendations. 

We expressed concern that an overall government 
eHealth strategy had still not been approved and 
that no timelines had been established for the 
development of the EHR. 

In July 2007, the Ministry developed a draft 
strategy that explored the possibility of developing 
the EHR by expanding an existing system for pedi- 
atric patients run by the Hospital for Sick Children. 
This system was called the electronic Child Health 
Network. The strategy called for delivery of the 
EHR in three waves: 

@ 2007-2010: By the end of this period, most 

Ontarians would have an EHR. Many patients 
would be going on-line to update their own 


data. Most clinicians would access patient 
information through the EHR. 

e@ 2011-2012: By the end of this period, the EHR 
would be accessible throughout the primary 
care system. Chronic disease patients would 
be able to interact with their providers on- 
line. Ontarians would be referred between 
providers electronically. 

@ 2014-2016: By the end of this period, all 
Ontarians would be able to work with their 
care provider on-line. Ontarians would 
proactively manage appointments, tests, and 
prescription renewals, and medical informa- 
tion would follow the individual throughout 
the health-care information system. 

This proposed strategy was never approved. 

One serious consequence of the absence of an 

approved eHealth strategic plan was that, even 
though SSHA spent $800 million building and run- 
ning a network (now run by the eHealth Ontario 
agency), this work was not well co-ordinated with 
ministry efforts. There were few clinical applica- 
tions available to enable the sharing of EHR data 
and few databases available to feed these applica- 
tions. Returning to the earlier analogy, there were 
few vehicles on the highway, and even those that 
existed were carrying few if any of the goods that 
the highway was built to carry. Although some of 
the broader EHR applications were begun earlier, 
many of the critical projects were not begun 

until 2008. The Ministry is responsible for this 
significant lag—until the formation of the eHealth 
Ontario agency, the Ministry was responsible for 
the eHealth strategy and ensuring that applica- 
tions and data of value to health-care providers 
were developed. An effective strategy would have 
planned for a more co-ordinated delivery of the 
network and the applications and data content, 

so that stakeholders would see the network’s 
value and want to be onit. In reality, however, the 
network today sits for the most part underutilized, 
with significant excess capacity, while time and 
technological progress erode its value and necessi- 
tate continual reinvestments to keep it available for 


when the vehicles and goods are eventually ready 
to travel on it. 

A second consequence of the absence of a 
strategic plan is that work on the EHR initiative 
proceeded without adequate planning for a means 
for all Ontario citizens to access the network. SSHA 
built a private network that is currently usable only 
by the health-care community. In the same way that 
it is inconceivable that every Ontarian’s driveway 
could be directly connected to an onramp of a 400 
series highway, the network itself can never grow 
large and intricate enough to provide the connec- 
tions that will enable all Ontarians to directly con- 
nect to it. Rather, other networks, and the Internet 
generally, will need to be used and connected to 
the network SSHA built. Since the Internet is for 
the most part an unregulated, open, and unsecure 
environment, appropriate technical safeguards 
need to be in place to protect the confidential data 
travelling through it. Interception of the data will 
be a constant concern, so providing these connec- 
tions while at the same time protecting patient 
security and the privacy of extremely personal and 
confidential data is a key obstacle to overcome in 
completing the EHR. 


The 2009 Strategic Plan 


In 2008, the Ministry engaged the services of a 
health-care consulting firm to help develop an 
eHealth strategy that would be acceptable to the 
government. This resulted in an August 2008 
ministry eHealth strategy submission to the govern- 
ment that sought approval for, among other things, 
“the establishment of an electronic health record 

of clinical information for every patient, and that 
could be controlled by the patient, by 2015.” The 
submission was approved and, once the eHealth 
Ontario agency was formed in fall 2008, an eHealth 
strategy document consistent with this submission 
was completed, publicly released for comment 

in January 2009, and published in final form in 
March 2009. 
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The Strategic Plan calls for the electronic health 
initiative to be implemented in phases, with three 
shorter-term deliverables identified as its top 
priorities: 

e@ The Diabetes Registry—The government 
estimates that diabetes afflicts approximately 
900,000 Ontarians, has increased by 69% 
over the last 10 years, and accounts for 3,200 
deaths annually. One of the Strategic Plan’s 
major initiatives is the creation of a Diabetes 
Registry. The Registry is to compile diabetes 
patient and health-care-provider data, enable 
electronic monitoring of adherence to best 
diabetes-management practices, and provide 
alerts to providers when best practices are not 
being followed. 

e The ePrescribing and Drug Information 
System—The Ministry reports that the 
fourth-largest cause of death in Ontario is 
preventable adverse drug reactions and that 
having an incomplete medication list is the 
primary source of medical error. The Strategic 
Plan calls for a system that allows doctors to 
prescribe medications electronically, gener- 
ates patient medication profiles, checks for 
allergy and drug-to-drug interactions, elimin- 
ates errors resulting from legibility problems, 
reduces dosing errors, and improves the 
management of complex therapies. It is also 
intended to provide patients and their families 
with their medication history and the ability 
to record and track personal medical data 
such as their blood-sugar readings. 

@ The Wait Times Initiative—Unlike the other 
two priorities, this initiative does not directly 
relate to the EHR. As the name suggests, it is 
aimed at reducing wait times for key health 
services and making wait times available to 
the public on a government website designed 
to track access to those health services. 

One aspect of the Strategic Plan that we par- 

ticularly welcomed was the robust and detailed 
description of activities to be conducted from 2009 
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through 2012. To its credit, the Plan sets out a num- 
ber of concrete targets and deliverables on each of 
the key EHR components. It thus represents a major 
step forward in crystallizing the government’s 
eHealth priorities and plans, and communicating 
these to stakeholders. 

That said, there are areas where we believe 
more strategic attention is required. The Strategic 
Plan envisions its three priorities as “quick-win” 
projects with immediate clinical value. Although we 
certainly understand the desire to complete shorter- 
term “quick wins” to gain health-care-provider 
support for the EHR agenda, we are concerned 
that this new focus means the goal of delivering 
an EHR by 2015 is being, in the Ministry’s words, 
“de-emphasized.” In other words, there is a risk 
that these three priorities are being emphasized at 
the expense of addressing how the eHealth Ontario 
agency will deliver a fully functional EHR for all 
Ontarians by 2015. 

Another area of concern related to the Strategic 
Plan providing details on the work planned through 
to 2012 only. By the end of 2012, there will still 
be considerable work to do to achieve a fully func- 
tioning EHR, even if every piece of work planned 
for in the Strategic Plan is completed on time and 
with its intended functionality fully achieved. The 
Strategic Plan does not indicate how close the gov- 
ernment will be in 2012 to the actual deployment of 
the EHR; nor does it specify what the major remain- 
ing pieces of the puzzle will be at that point in time, 
how much work they will require, and what the 
government plans or will need to do to complete 
this work over the following three years. 

We specifically raised this issue when we met 
with the consulting firm engaged to help draft 
the Strategic Plan. In response, the firm provided 
us with a May 2009 document it had prepared to 
outline the gaps it expected would remain in 2012 
once work under the Strategic Plan is completed. 
This document outlined the activities expected to 
be incorporated into the next eHealth Strategic 
Plan covering the period from 2012 through 2015 


to close these gaps. However, this document has 
not been released to stakeholders or the public and 
does not have any official or approved status with 
the government. 


RECOMMENDATION 1 





The eHealth Ontario agency should develop a 
comprehensive strategic plan that specifically 
addresses the Electronic Health Record (EHR) 
target, takes Ontario beyond 2012, and lays 
out the path for the implementation of the 
EHR by 2015. The plan should also address the 
challenge of ensuring that the disparate EHR 
applications are appropriately integrated and 
that suitable privacy controls are built into the 
development process. 





In 2007, the government announced that it 


would develop an Electronic Health Record for 
all Ontarians by 2015. The 2009 Ontario Budget 
also provided approximately $2 billion over the 
next three years to allow the province to move 
ahead in a co-ordinated and meaningful way. 

Ontario’s plan towards an EHR has three 
components: building a technology infrastruc- 
ture; developing clinical applications; and creat- 
ing the EHR itself and its supporting technology. 

Ontario has directed its efforts toward all 
three areas. As have other jurisdictions, Ontario 
has decided to use a specific clinical priority 
—diabetes—as a focus point in the creation of 
an EHR. Achieving EHRs for diabetes patients 
will provide a foundation for achieving EHRs 
for patients with other chronic diseases. From 
there, EHRs can be extended to all Ontarians. 

The Ministry recognizes that more work is 
required to meet the government’s commitment 
to an EHR for all Ontarians by 2015, and will 
continue to work with eHealth Ontario to meet 
this commitment. 





Over the last number of months, eHealth 


Ontario has completed its detailed business 
plan, which relates Ontario’s eHealth Strategy 
2009-2012 to the daily activities of the organ- 
ization that are under way in support of system 
development and operations. Foundational 
priorities in the Strategic Plan that are essential 
to the Electronic Health Record are integral to 
this plan. 

With this important piece of work com- 
pleted, the organization, under the leadership 
of the board of directors, will now commence a 
process to develop a plan that takes the organiz- 
ation beyond 2012. This plan will lay out a road 
map that specifies deliverables, risks, timelines, 
privacy measures, and how the various applica- 
tions will be integrated. 

This planning process will have to be a col- 
laborative one between eHealth Ontario and 
the Ministry of Health and Long-Term Care. It 
will also require the active engagement of key 
stakeholders across the province. 


OVERSIGHT 


Ministry and SSHA Roles and 
Responsibilities 


Most of the public criticism for the lack of progress 
on the EHR initiative has been directed at SSHA. In 
our view, this is not entirely fair. We clarify here the 
roles and responsibilities of the Ministry and SSHA 
with respect to the EHR agenda. 

SSHA was established in 2002 as a provincial 
agency by regulation under the Development Cor- 
porations Act. Its initial mandate was to support 
specific ministry programs, but the regulation was 
amended in 2005, and SSHA’s mandate was clari- 


fied as being to: 
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e provide and operate secure infrastructure 
to enable secure transfer of personal health 
information among sector providers; and 

e host personal-health-information applica- 
tions, including core provincial systems that 
are the source of information for the EHR. 

In September 2007, the Ministry and SSHA 
signed an Affirmation of their Memorandum 
of Understanding that reinforced their mutual 
understanding of their respective roles and respon- 
sibilities. It indicated that SSHA’s mandate was 
to provide “the secure, integrated, province-wide 
information technology infrastructure to allow 
electronic communication among Ontario’s health 
service providers.” The components of this infra- 
structure were: 

@ a managed private network; 

e security infrastructure; 

e physical facilities; 

@ secure messaging service and on-line 

directories; 

e data and technology standards; and 

e a Voluntary Emergency Health Record 

(to provide health-care providers with critical 
personal health information in times of 
emergency). 

SSHA’s Accountability Agreement with the 
Ministry for the 2007/08 to the 2010/11 fiscal years 
contained the Ministry’s performance expectations 
for SSHA for just the 2007/08 fiscal year. These 
expectations also made it clear that the Ministry 
saw SSHA as an IT infrastructure provider and not 
a strategic partner in delivering the full eHealth 
agenda. Except for some deliverables relating to 
recommendations arising from two external reviews 
(one by Deloitte Consulting in 2006 and the other 
by the Information and Privacy Commissioner in 
2007), the deliverables were technical in nature and 
reflected existing plans to expand the network. 

The Ministry’s eHealth Program Branch mon- 
itored SSHA’s operations and approved its budgets. 
The Ministry was also responsible for developing 
the overall eHealth strategy and setting SSHA’s 
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direction and business priorities. Of particular sig- 
nificance were its responsibilities for co-ordinating 
the development of the applications and databases 
that would enable health-care practitioners to share 
health-care data on SSHA’s information highway. 
The lack of an approved strategic plan until 2009 
undoubtedly contributed to the Ministry’s inability 
to effectively deliver on these responsibilities. 

It has been said that shared accountability 
means no accountability; the relationship between 
the Ministry and SSHA may be a textbook example 
of this. By all accounts, it was not a true partner- 
ship and was marred by a lack of collegiality and 
confusion over each party’s respective roles and 
responsibilities. The Ministry and SSHA blamed 
each other for many of the failures and delays in 
system implementation. 

A telling example of how the Ministry was not 
working hand in hand with SSHA involves key 
SSHA employees leaving SSHA to work for the 
Ministry. Former SSHA executives advised us that 
they often faced operational problems as a result 
of employees moving to the Ministry to take con- 
sulting contract positions at significantly higher 
compensation levels. SSHA’s board minutes support 
this complaint; in one of his reports to the board, 
the SSHA Chair noted: 


The [Ministry’s eHealth Program Branch] 
has 10 [to] 11 people that came from 
SSHA. Some were forced out of SSHA 
because of contract versus employment 
issues, but others left because they were 
aware of lucrative contracts at the [Min- 
istry’s eHealth Program Branch]. Several 
others were recruited by the [Ministry’s 
eHealth Program Branch]. In some 
cases, staff that were employees at SSHA 
became contractors at [the Ministry’s 
eHealth Program Branch]. 


Ministry Oversight of SSHA 


From its inception, SSHA grew rapidly, with its 
spending on EHR initiatives increasing from 
$13.6 million in its first year of operations 
(2002/03) to $213 million in the 2008/09 fiscal 
year. In total over this period, SSHA spent more 
than $800 million, primarily on building a secure 
data-transmission network, connecting many of 
the estimated 24,000 publicly funded health-care 
sites in Ontario to this network, and working to 
provide the estimated 150,000 people working in 
the health-care system with email accounts on this 
network. 

We assessed the oversight of SSHA operations 
and spending by the Ministry’s eHealth Program 
Branch and saw little evidence that this monitoring 
responsibility had been adequately fulfilled. 

SSHA was required to provide monthly financial 
reports and quarterly performance reports to the 
Ministry's eHealth Program Branch, and we asked 
to review these. The Ministry’s eHealth Program 
Branch could locate only the February 2009 
monthly financial report. The reports for all other 
months from 2003/04 through 2008/09 could not 
be located. Ten of the 23 quarterly performance 
reports for the same six-year period could also not 
be located. 

With the formation of the eHealth Ontario 
agency, the Ministry began rebuilding its oversight 
unit, and the organizational structure of this unit 
has yet to be finalized. 


eHealth Ontario Agency Governance 


To a large extent, the sole-sourcing of a number 
of significant consultant contracts at the eHealth 
Ontario agency is an indication of financial govern- 
ance problems at the agency. The agency’s first CEO 
wielded considerable power to make unilateral 
decisions and essentially ignored procurement poli- 
cies. There are a few possible reasons for this: 
e The board did not adequately oversee the 
CEO’s actions and decisions in this area. It 


appears to have accepted the CEO’s assertion 
that government and agency procurement 
policies were being followed, without getting 
sufficient details as to what that meant. For 
example, it never asked whether any contracts 
were sole-sourced and, if so, why, and what 
the value of such contracts was. 

At the same time that a great deal of power 
was concentrated in the CEO’s hands, the 
agency was faced with the challenge of 
accelerating progress on the EHR initiative, 
considered by many to have suffered when 

it was managed by SSHA and the Ministry’s 
eHealth Program Branch. It was during this 
period, from October 2008 to June 2009, that 
millions were paid to consultants who had not 
been competitively procured. In other words, 
it was during this time of transition that the 
CEO and her executive management overrode 
the normal procurement and contract man- 
agement controls that SSHA’s procurement 
department had in place. Our discussions 
with the CEO indicated that urgency and the 
implied authority she thought she had been 
given justified and warranted this. 

The Chair told us that the Premier had person- 
ally asked him to chair the eHealth Ontario 
agency. He further informed us that he had 
told the Premier that his acceptance of the 
position was largely conditional on his being 
able to choose the CEO and the agency having 
the autonomy to act without significant gov- 
ernment interference. We were advised that 
the Premier met with the Chair’s choice for 
the new CEO to impress upon her the import- 
ance of her upcoming work. Given that it had 
no input into the appointment of the CEO, 
and understanding that the Chair had person- 
ally chosen the CEO with the support of the 
Premier, the board may have had a perception 
that its oversight role with respect to the CEO 
was more limited than it would be in normal 


circumstances. 
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The board’s approval of a performance bonus 
for the CEO midway through her first year on the 
job is an example of the extent to which the board 
may have felt it had little power to oversee matters 
relating to the CEO or question how she should be 
treated. There are two issues with respect to this 
bonus—the amount paid and the fact that it was 
awarded after only four months on the job. 

The amount of the bonus paid to the CEO 
was $114,000, which was 30% of her salary. The 
maximum bonus rate under eHealth Ontario policy 
was 15%. However, the CEO had negotiated her 
30% maximum bonus as one of the conditions of 
accepting the job, and it was included as part of her 
employment contract, approved by the Minister and 
authorized by an Order-in-Council. Although the 
bonus was significantly higher than what govern- 
ment Deputy Ministers can receive, we recognize 
that the government has to have some latitude 
with respect to executive compensation. The board 
therefore had no say with respect to the bonus 
available. It did have a responsibility, however, to 
assess whether the CEO’s performance justified the 
full 30% and whether the full bonus was warranted 
only four months into the job. 

We noted that the board approved payment of 
the $114,000 annual bonus on March 4, 2009. We 
were advised that the Chair of the Board recom- 
mended that the CEO be paid the full amount of the 
annual bonus because, in addition to having per- 
formed in an exemplary manner, she had foregone 
the bonus she could have earned as of March 31, 
2009, if she had continued to be employed by her 
former employer, Cancer Care Ontario (CCO). 
However, the maximum allowable bonus at CCO 
was 15%, and in the previous year at CCO she had 
received a much smaller bonus of $38,000. 

Although we were advised that the board did 
discuss the CEO’s bonus, we saw nothing in the 
board minutes to indicate that the board formally 
questioned whether awarding the CEO the full 
amount of the annual bonus after only four months 
was appropriate. 
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RECOMMENDATION 2 


To ensure that governance arrangements for the 
eHealth Ontario agency allow the government 
and the agency’s board of directors to exercise 
informed oversight while allowing management 
the day-to-day operational autonomy needed 

to fulfill the agency’s mandate, the following 

three accountability mechanisms should be 

implemented: 

@ The eHealth Ontario agency’s board of direc- 
tors should clearly specify the performance, 
operational, and financial information they 
need to obtain regularly from agency senior 
management to enable them to meet their 
oversight responsibilities. 

e Specific targeted goals or benchmarks for the 
agency, including timelines and estimated 
costs, should be established and regularly 
reviewed to enhance the oversight of agency 
operations by the agency’s board and the 
Minister of Health and Long-Term Care. 

e Progress towards these targeted goals— 
along with details of major issues, risks, 
and explanations for variances—should be 
succinctly communicated in writing to, and 
discussed in periodic briefings with, the 
agency’s board and the Minister of Health 
and Long-Term Care. 





The Ministry of Health and Long-Term Care and 


the Smart Systems for Health Agency have had 
an accountability relationship since 2003. In 
April 2009, the Ministry and eHealth Ontario 
signed a Memorandum of Understanding and 
a Transfer Payment Accountability Agreement. 
These documents provide clear accountability 
and reporting frameworks. 

The Ministry has recently formed an eHealth 
Liaison Branch within the Ministry to formally 
oversee eHealth Ontario. The Ministry and 
eHealth Ontario are eager to work together to 





ensure that proper procurement policies and 
procedures are in place and that regular compli- 
ance monitoring is undertaken. 





Building on the annual business plan, the board 


approved a Balanced Scorecard Framework to 
ensure that management will provide quarterly 
status reports on the progress of initiatives to 
the board. These reports will include perform- 
ance against specific targeted goals, financial 
targets, project timelines, and outcome meas- 
ures. The scorecard has been developed not 
only as a mechanism for the board to provide 
stewardship and oversight, but also as a com- 
munications tool to government and the public. 

At the September board meeting, the board 
also approved a policy on risk management 
that clarifies its oversight responsibility for risk 
management. 


PROGRESS ON EHR PROJECTS 


In this section, we discuss the application and data 
projects (for example, registries, information sys- 
tems, and portals) that make up the EHR initiative, 
their status, and project-management issues that 
will need to be proactively managed if Ontario is to 
meet its commitment of completing the EHR initia- 
tive by 2015. 

Canada Health Infoway tries to ensure that the 
separate EHRs being developed in each jurisdiction 
are compatible across Canada. It also issues tech- 
nical standards, guidelines, and toolkits, and funds 
up to 75% of the cost of provinces’ and territories’ 
approved projects. The Infoway has been monitor- 
ing the progress of Canadian provinces and territor- 
ies in implementing EHRs, focusing its assessment 
on the following six key components: 

e A Client Registry—Each jurisdiction is 

attempting to develop a single patient direc- 
tory that will contain, for each member of 


the population, a unique health identification 
number, along with current and historic 
demographic information such as the indi- 
vidual’s name, address, and birth date. It is 
critical to the success of the EHR initiative 
that the information in this registry be com- 
plete, accurate, and updatable, and, further, 
that the registry be designed to ensure that 
each individual’s information remains strictly 
confidential. 

A Provider Registry—Each jurisdiction is 
attempting to develop a single health-care- 
provider directory that will contain a full 

list of health-care professionals (including 
physicians, dentists, pharmacists, nurses, and 
others) authorized to access the EHR system. 
A Diagnostic Imaging System—Each jurisdic- 
tion is attempting to develop an information 
system that will collect and store all diagnostic 
images such as Magnetic Resonance Imaging 
(MRI), Computed Tomography (CT), X-ray, 
and ultrasound test results entirely in digital 
format, without the need for film. The system 
is meant to allow practitioners to manage, 
distribute, and view these images regardless 
of where they are located or where they were 
obtained. 

A Drug Information System—Each jurisdic- 
tion is attempting to develop an information 
system that will allow health-care providers 
to access, manage, and share their patients’ 
medication history, including prescribed 

and dispensed drugs, allergies, and ongoing 
drug treatments. It is hoped that the system 
will enable drug-interaction checks to be 
performed automatically to assist physicians 
and pharmacists with their prescribing and 
dispensing decisions and to help reduce the 
number of adverse drug interactions. 

A Laboratory Information System—Each 
jurisdiction is attempting to develop an 
information system that will allow authorized 
practitioners to order laboratory tests for their 
patients and to view the results of these tests 
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electronically from their offices regardless of 
where the test was conducted. 

@ Clinical Reports or Immunization—Additional 
client and practitioner information systems, 
for example, an application to deal with 
immunizations, fall into this sixth category. 

In its 2007/08 corporate business plan, the 
Infoway ranked Ontario ahead of the territories 
and the provinces of Nova Scotia, Quebec, and New 
Brunswick, but behind all six remaining provinces. 
It further projected that Ontario will have slipped 
behind the provinces of New Brunswick and Que- 
bec by March 31, 2008, and thus only be ahead of 
the Northwest Territories, the Yukon, and Nunavut. 
Figure 2 shows the Infoway’s latest assessments of 
jurisdictional progress as of March 31, 2009. The 
Ministry indicated that the Infoway’s approach 
to measuring EHR progress might not necessarily 
reflect all the work required to develop an EHR, 
especially in a large province like Ontario. However, 
we believe the Infoway’s assessment provides a 
useful benchmark. As well, although Figure 2 indi- 
cates that Ontario’s Client Registry and Diagnostic 
Imaging System are both 95%-100% complete, our 
work indicated that these systems are likely not that 
far along in their development. We discuss this in 
more detail later in this section. 

One factor contributing to Ontario’s slippage has 
been that the data and application projects were 
delayed from October 2008 through June 2009, 
while the eHealth Ontario agency was getting 
established. A number of EHR projects were put 
on hold awaiting clarification of their scope and 
direction, and revised project charters for all EHR 
projects were not completed until July 2009. This 
has contributed to the pushing back of a number of 
expected completion dates for various projects. 

Although Ontario’s EHR projects partially align 
with those identified by the Infoway, the Ministry 
is working on a number of other applications and 
registries, several of which are essential to the even- 
tual realization of an EHR. These include: 

e Electronic Medical Record Systems—World- 

wide, primary-care physicians are automating 
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Figure 2: Jurisdictional Progress on Six Key EHR Deliverables as of March 31, 2009 


Source of data: Canada Health Infoway 


Information System 


Jurisdiction 





95%-100% complete 
partially complete 
planning under way 
forecast 


their patient records (including demograph- 
ics, medical and drug history, and other 
information such as laboratory results and 
findings from diagnostic imaging). Systems 
for automating patient records are called 
“Electronic Medical Record” (EMR) systems. 
It is recognized that information from these 
systems will be the primary source for some 
of the significant data to be captured in the 
eventual patient EHR. 

e A Consent Directive and Privacy Audit Applica- 
tion—This will include a registry of patient- 
consent data and an access log to maintain 
information on who has accessed data from 
a patient’s EHR. This will be a particularly 
important system for protecting patient pri- 
vacy and confidentiality, because it will give 
individuals some control over their personal 


health information and allow them to monitor 
who is accessing it. 


e A Diabetes Registry—This registry is meant to 


compile diabetes patient and provider data, 
enable electronic monitoring of adherence to 
best diabetes-management practices, and pro- 
vide alerts to physicians when best practices 
are not being followed. 


e An application that integrates all other applica- 


tions and links them to a portal—This will 
integrate the key applications that have been 
created so that they can communicate and 
share patient-record information with each 
other “behind the scenes.” Once integrated, 
they must be linked to a central portal or 
“viewer” to enable authorized health-care 
providers to view patient data residing in a 
number of different databases on one screen, 
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as illustrated in Figure 1. This will also enable time and, as yet, delivering the required functional- 


health-care providers to update patient infor- ity to meet user needs. Projects were also typically 
mation on-line through a single interface. This started before all of the resources required to com- 
end product will in essence be a “one-win- plete them had been identified and procured. For 
dow” view of all the key clinical and personal example, a complete analysis of staff and consultant 
health information for a patient. hours and other input costs, such as the hardware 
e Animplementation and adoption project—This _ or software acquisitions needed to complete the 
project is meant to support the successful project, were often not determined. Once projects 
implementation and adoption of all the had started, project expenditures and deliverables 
applications and information systems across were not being adequately tracked to give manage- 
Ontario’s 14 Local Health Integration Net- ment assurance that the deliverables were being 
works (LHINs) and related communities. attained as the related costs were being incurred. 
Figure 3 summarizes total EHR expenditures We also found that the whole issue of integrating 
to March 31, 2009—where possible, we have all these systems and addressing the privacy issues 
indicated amounts allocated to the various EHR for each will require significantly more attention. 
component projects. The following sections outline our observations 


on the most significant EHR projects under way. 


Project-management Concerns 

The EHR Network and Related Products 

and Services 
The network and the two data centres that were to 
underpin the EHR were developed by SSHA and 


We found few of the information-system and 
registry projects to be progressing well from the 
standpoint of completing their various phases on 


ee 


Figure 3: Expenditures! on EHR Projects and Initiatives ($ million), 2002/03-2008/09 


Source of data: Ministry of Health and Long-Term Care 





Project/ Initiative 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 Total 




















SSHA? 12 85 87 11) 125 We 224 817 
Ministry’s eHealth Program - 3 3 4 5 9 5 6 
Branch 

Diagnostic Imaging System _ - 10 10 1 23 2 69 
Drug Information Systems - — : ri 3 - 4 18 
Laboratory Information = : 8 714 28 39 7 96 
Systems 

electronic Child Health _ - - e: 2 9 9 18 
Network? 

Identity Access and Privacy _ - - - - — 19 19 
Diabetes Registry — — _ — _ — 5 2 
integrating all systems and a © 7 - a 10 10 
linking to a portal 

implementation and adoption - — = - - 11 11 






1, These expenditures include overhead, salary, and administrative spending in additiion to spending on EHR projects. 

2. SSHA expenditures include spending on the network. 

3. The electronic Child Health Network is an EHR system for children under 19 years of age that the Ministry had planned to expand in 2007 but for which 
future plans are now unclear under the new 2009 eHealth Strategic Plan. 
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are now the responsibility of the eHealth Ontario 
agency. As of December 2008, the data centres 
housed an estimated 1,300 servers, and the net- 
work connected some 3,500 clients. Clients include 
all Ontario’s public hospitals, public health units, 
community care access centres, and retail chain 
pharmacies; many of the province’s continuing-care 
organizations; and some physician offices. Network 
services were outsourced to two vendors that are 
providing the circuits connecting the medical com- 
munity to the network. SSHA did not adequately 
manage these vendors—for example, penalties 

for poor performance were not built into one of 

the service-level agreements, which impaired the 
government’s ability to ensure that appropriate 
performance standards were met. 

Throughout the first half of 2008, problems with 
the data centres resulted in services often being 
unavailable to clients. The problem was particu- 
larly acute in May, with service outages occurring 
intermittently over a period of two weeks. In an 
August 19, 2008, report to the board, SSHA’s Chair 
noted that there had been five significant service 
interruptions since January. Upon investigation, 
it was determined that most of the incidents were 
caused by issues with the data centres rather than 
with the broader network. We were advised that 
the problem was fixed, and service disruptions were 
significantly reduced by the second half of 2008. 
However, a consulting firm hired to review three of 
the most significant service interruptions and make 
recommendations to restore stability concluded 
that the network continued to be at high risk of 
incurring significant business outages because of: 

e a lack of standards and operational processes; 

e conflicting priorities (focus on expanding 

the network at the expense of operational 
performance); 

@ immature capacity and performance 

capability; 

e the absence of an infrastructure lifecycle- 

management process; 

e a lack of meaningful metrics to monitor sys- 

tem performance; 


@ an aging network core approaching the end of 
its useful life (the core was built in 2002). 

The 2009 eHealth Strategic Plan acknowledges 
that “major portions of the infrastructure are 
out of date and prone to failure” and notes that 
the arrangements with the two outsourced firms 
operating the network are not being adequately 
managed. 

An SSHA product developed for the network 
is its email service, called “ONE Mail,” built at an 
estimated cost of $16 million. At the time of the 
eHealth Ontario agency takeover, ONE Mail had 
225 organizational clients and 120,000 individual 
clients, and it was targeting annual expenditures 
of $4.4 million for additional deployments. It is 
interesting to note that, as early as 2006, only 
approximately one-third of ONE Mail accounts 
were estimated to be in active use, and 53% of 
clients responding to a survey said they were not 
satisfied with it. When the eHealth Ontario agency 
was formed, it assessed ONE Mail as being inferior 
to commercially available email offerings and 
assessed ONE Mail’s dependency on the SSHA 
network as a fatal flaw—whenever there was a 
network outage, all email accounts were impacted. 
ONE Mail deployments were also significantly 
behind schedule. 

We found the network to be both expensive to 
operate and significantly underutilized, with no 
effective cost containment strategy in place. Of the 
$72 million in operating costs incurred annually to 
maintain the network, $30 million (or $2.5 million 
per month) was being incurred to provide com- 
munication lines that carry little or no traffic. We 
analyzed traffic on the entire network, covering the 
circuits provided by both vendors, and observed the 
following: 

e For the most part, the network is simply not 
being used. On average throughout the net- 
work, clients are using only 0.5% of the avail- 
able bandwidth, and the average peak usage 
is only approximately 16 % of the available 
bandwidth. 


e Even though vendors were supposed to mon- 


itor network traffic, they were not identifying 

individual circuits that were inactive or under- 

utilized. Monitoring was limited to identifying 
only the busiest circuits to assess whether 
they should be upgraded. Of 4,178 circuits, 
we estimated that only 116 may require such 
upgrades because peak usage was exceeding 

70% of the available bandwidth. The current 

monthly costs associated with these particular 

highly utilized circuits is only $90,400, or less 
than 2% of the network’s monthly cost. 

The network’s administrators have an “uplift 

process” to increase bandwidth to clients 

when required. In our view, there is therefore 
no reason to provide excessive bandwidth in 
advance of need. We found, however, that out 
of the 4,178 circuits deployed, 1,386 circuits 
are totally inactive, almost inactive, or sig- 
nificantly underutilized. The eHealth Ontario 
agency is paying more than $2.5 million per 
month to maintain these circuits—more than 
half the total monthly cost of operating the 
network. Specifically: 

e 198 circuits had absolutely no traffic. 

e 782 circuits were almost inactive, with an 
average peak utilization of less than 1% of 
the available bandwidth. The traffic we did 
note on these circuits was so sparse it may 
well have been generated by the network’s 
own monitoring software. 

e 406 circuits were significantly underutil- 
ized, with an average peak utilization of 
less than 10% of the available bandwidth. 

We further found that the devices that 

compiled and provided data to the network’s 

monitoring and reporting software had been 
configured incorrectly and were not produ- 
cing accurate data. Specifically, the system 


was overstating the usage of 405 of its circuits. 


Without accurate usage data, management 
cannot appropriately address utilization 
issues. 
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We also noted that, in 2001, consistent with 
federal-government and industry standards for 
security, SSHA acquired 150,000 Public Key Infra- 
structure (PKI) certificates (tools for ensuring that 
users can securely communicate with each other 
and be sure of the identity of the party they are 
interacting with), planning to provide them to all 
end users. However, this approach to secure com- 
munications was abandoned after the purchase, 
with only an estimated 2,500—3,000 PKI certificates 
being issued. Despite this, the agency continues to 
pay an annual licence fee to maintain these certifi- 
cates. Some $2.4 million in such fees had been paid 
at the time of our audit. 


The Client Registry 
As mentioned, a complete and accurate Client 
Registry is considered an essential precondition to 
the creation of EHRs. The cost of developing the 
Client Registry is estimated at $44.8 million. 

Both SSHA and the Ministry have struggled with 
an appropriate approach to developing the Regis- 
try. When we first reviewed SSHA’s operations, the 
idea of building the registry using Ontario Health 
Insurance Plan (OHIP) records as a base had been 
rejected because of the incompleteness of these 
records and because the data were considered to 
be of questionable reliability. However, developing 
the registry via an independent process proved too 
challenging, and the current plan calls for build- 
ing the Client Registry from OHIP’s Registered 
Person Database, which includes all Ontarians 
currently covered by OHIP. The first release of the 
Client Registry was planned for June 2009, but this 
has been delayed partly because of difficulties in 
reaching a data-sharing arrangement with OHIP 
agreeable to both the Ministry and the agency. The 
data quality of the Registered Person Database 
also remains an issue, and the identification of the 
approximately 4% of Ontarians who are not cov- 
ered by OHIP has yet to be resolved. 

Some milestones originally scheduled for 
completion in 2009/10 have been pushed back 
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one year, and the detailed timelines for deliver- 
ables have been revised. Only one Client Registry 
deliverable is now slated for 2010/11 and none for 
2011/12. These setbacks are of concern because 

a fully functional and secure Client Registry is an 
essential building block in the development of a 
province-wide EHR system. 


The Diagnostic Imaging System 
This project essentially started when the Ministry 
began supporting regional initiatives aimed at 
migrating from film and paper formats for diag- 
nostic images (such as x-rays, ultrasound images, 
and CT and MRI scans) and their associated reports 
to a digitized format and storing them in regional 
repositories. The project now aims at making the 
digital images and reports available to health-care 
providers at the point of patient care. Ultimately, 
the regional repositories are to be expanded and 
integrated into the provincial EHR. At the time of 
our audit, there were four regional repositories in 
place that allowed for the storage, retrieval, distri- 
bution, and presentation of diagnostic images. A 
key project target is that 100% of diagnostic images 
taken by hospitals will be digitally stored and share- 
able among health-care providers by fall 2011. 

The first half of this key target has been met, 
and efforts at enabling image-sharing are ongoing. 
Total project costs are projected to be $280 million, 
$107 million of which it is hoped will be provided 
by Canada Health Infoway. The initial targeted 
completion date for the system of March 2009 has 
been pushed back several times. For example, when 
we began our audit, the expected completion date 
had been set back 18 months to September 2010, 
when it was decided to consolidate the original 
plan for six repositories and instead develop four 
repositories to realize project savings estimated at 
$14 million. We also noted at the time of our audit 
that project plans did not include having some 
700 independent health facilities linked to the 
repositories—even though 40% of the province’s 


diagnostic images are currently maintained by 
these facilities. 


The Drug Information System 
The Drug Information System is the second of three 
priorities in the eHealth Ontario agency’s 2009 
Strategic Plan. eHealth Ontario expects that, if it is 
delivered as planned, a medication-management 
system has the potential to save an estimated 
$350 million annually in health-care costs. 

The Strategic Plan describes four activities being 
undertaken as part of this project: 

e@ a demonstration of electronic prescribing; 

e@ expansion of an existing drug-profile viewer; 

@ acomputerized medication-ordering system 

for oncology clinicians; and 

e the procurement of a drug-information 

system to provide clinicians with a medica- 
tion profile of all Ontario residents and their 
pending prescriptions and to provide patients 
with their medication history and the ability 
to record and track certain medical data such 
as blood-sugar readings. 

The project is planned to be completed over 
four years, at a total cost of $150.3 million. The 
first activity, the demonstration of electronic pre- 
scribing, has been completed. Next, the existing 
drug-profile viewer is to be expanded from its 
current availability in hospital emergency depart- 
ments. Then, patient medication profiles are to be 
developed by capturing dispensing events from 
community pharmacies. Physicians are then to be 
connected to the system and, finally, other health- 
care providers are to be connected. 

We also noted some slippage on this project. 
Installation and testing targets scheduled for April 
2009 had not been met, and work on the capture 
of dispensing events, slated to begin in April 2010, 
has been delayed. Infrastructure Ontario will be 
responsible for procuring the drug-information sys- 
tem, which is to contain a full medication profile of 
Ontarians. The eHealth Ontario agency expects this 
procurement to take place in October 2010. 


The Laboratory Information System 
This project is aimed at the development of a cen- 
tralized repository of laboratory test results that 
will allow authorized practitioners from across the 
province to both order and view laboratory test 
results electronically. This project actually began 
in the late 1990s, and it now collects and stores a 
large and growing proportion of test results from 
hospitals and commercial labs. 

If completed as planned, clinicians will be able 
to view on-line laboratory results and reports from 
all hospital, community, and public health labora- 
tories regardless of where the test was conducted. 
The results are to be linked to, and form part of, 
patients’ EHRs. It is expected that a fully functional 
system will reduce unnecessary duplicate tests 
and help speed diagnosis and access to appropri- 
ate care. Some $96 million had been spent on the 
system at the time of our audit, and $210 million in 
total is expected to be spent by 2012. This amount 
includes the costs of implementation and adop- 
tion as well as the costs of ongoing operations and 
maintenance. 

In our review of this project, we found that it is 
not yet working as intended, particularly because 
statutory authority enabling the sharing of data 
is currently lacking. Our specific concerns were as 
follows: 

e Although the original project approval 
projected that 80% of laboratory test results 
would be recorded in the system by March 31, 
2009, only 51% were actually being recorded 
by that time. In addition, the original project 
approval called for 60% of the province’s lab- 
oratory test results to be accessible by users by 
that date, but none are as yet accessible. 

e The system was developed primarily by a 
consulting firm, which delivered its final prod- 
uct to the Ministry in August 2007. However, 
at that time both the Ministry and the firm 
realized more functionality was required to 
meet the needs of medical practitioners, and 
a number of gaps and defects in the system 
were identified. During the warranty period, 
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the Ministry focused its efforts on having 

the firm develop some of this additional 
functionality rather than fix product defects. 
In March 2008, when the warranty expired, 
the system still had 139 identified defects, 40 
of which were classified as “requirements” 
defects because they were associated with 

the attempts to add more functionality to the 
original design. Another 72 were considered 
major defects. The Ministry advised us that 

it, rather than the consulting firm, had to 
address almost all of them. 

The system, although technically ready to 

do so, was not yet providing any data to 
clinicians. Also, some of its deficiencies were 
immediately apparent. Although the database 
had correct data, the viewer often showed 
errors or blanks, its response times were slow, 
and at times it displayed the data incorrectly. 
Early users at hospitals and community labs 
were experiencing a range of difficulties with 
the interface, including outright application 
failure, non-responsiveness, and formatting 
problems. The Ministry was aware of the poor 
quality of the system’s viewer and informed 
us that it would not have released it to users 
if it had been able to develop a better one in 
time. But because it did not want to delay the 
current validation work being done at hospi- 
tals and community laboratories, it deployed 
an interface that it called the “viewer of last 
resort.” 

A key objective of the project is for community 
laboratories, hospitals, and other service pro- 
viders to be able to share test results regard- 
less of where the test was conducted. No data 
are currently being shared, however, because 
the consent management protocols and prac- 
tices needed to enable such sharing while pro- 
tecting patient privacy and confidentiality are 
not in place. As a result, the laboratories and 
hospitals participating in the early adopter 
project are doing little more than feeding data 
into the system and occasionally using it to 
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check their own data. The system is not giving 
them any access to laboratory tests conducted 
elsewhere. 

e The provincial roll-out of the system, origin- 
ally scheduled for fall 2007, is significantly 
behind schedule. Project staff are still working 
with a group of three community laborator- 
ies and four hospitals to complete an early 
adopter project that was originally expected 
to be completed in March 2008. 

@ The deliverables for the laboratory informa- 
tion system are still evolving, even though the 
system itself has been developed and is now 
in the implementation and adoption phase. 
Two significant recent deliverables added to 
the project were enabling patient access to 
the system and integration with the Diabetes 
Registry, both of which were not originally 
contemplated. 


Electronic Medical Record Systems 
In a 2006 Commonwealth Fund Survey of seven 
countries, Canada had the lowest percentage of 
primary-care physicians with EMRs (23%), with 
only 6% fully utilizing the EMR’s potential. The 
Infoway reports similar data, estimating that fewer 
than 20% of Canada’s primary-care doctors have 
implemented an EMR system to help manage their 
practice. Although the use of EMR systems is on the 
rise (National Physician Survey results for 2004 and 
2007, respectively, indicate that the percentage of 
Canadian physicians using an EMR system has risen 
from 18% to 26%), it is nowhere near the levels 
required for the EHR initiative to be fully realized. 

In May 2008, the Ontario College of Family 

Medicine released a discussion document that 
expressed concern over the lack of an eHealth strat- 
egy to ensure that EMRs will integrate effectively 
with other EHR initiatives. In its view, a key driver 
of efficiencies from EMRs is interoperability. It also 
reported that many physicians were unwilling to 
adopt EMRs because they did not think the network 
that SSHA built was reliable. 


The Ministry indicated that it recognizes that 
not enough physicians have adopted EMRs and that 
it has been working to improve the adoption rates 
for some time. In 2005, in partnership with the 
Ontario Medical Association, it provided funding 
to enable an estimated 30% of Ontario’s primary- 
care physicians, or 3,300 of them, to acquire EMR 
systems. This made Ontario’s EMR penetration rate 
comparable to the average rate of other Canadian 
jurisdictions. 

More than 1,700 additional primary-care 
physicians had applied for funding by the time the 
government’s initial EMR funding ran out in April 
2009. The board of the eHealth Ontario agency 
recently approved another $100 million in EMR 
support to continue this initiative. The eHealth 
Strategic Plan targets a 65% EMR adoption rate by 
primary-care physicians by April 2012, which it pro- 
jects will require further funding of $50 million in 
2011 and $77 million in 2012. Achieving the target 
is thus expected to cost more than $225 million. 

We acknowledge that primary-care physicians 
are the most logical current target for EMR adop- 
tion strategies. However, there are an estimated 
12,700 practice-based physicians (representing 
close to 56% of the province’s physician popula- 
tion) who do not have an EMR system. This 
includes practice-based specialists who are cur- 
rently excluded from applying for funding to help 
them acquire EMR systems. It is believed that some 
5,000 of them will eventually require such systems. 

An estimated $28,000 subsidy per physician is 
available under the current EMR funding model. 
The subsidy covers an estimated 75% of the costs 
of EMR software and software maintenance, office 
equipment such as computers and printers, secure 
network connectivity, and EMR training. If all 
practice-based physicians are eventually supported 
under this or a similar program, the total cost to the 
government of supporting physician adoption of 
EMRs will approach $450 million. 


The electronic Child Health Network 

The electronic Child Health Network (eCHN) is an 
EMR system originally developed in 1999 under 
the direction of the Hospital for Sick Children. 
It provides EMRs for patients under 19 years of 
age, and by the 2005/06 fiscal year, it was in use 
at 64 Ontario hospitals. That same year, Ontario 
launched a project to expand eCHN’s availability to 
key pediatric hospitals, community care access cen- 
tres, children’s treatment centres, physicians, and 
other health-care professionals. For each patient 
in its records, eCHN records and stores admission, 
discharge, and transfer information for all hospital 
encounters, laboratory test results, diagnostic 
image reports, other reports (such as those arising 
from visits to clinics), and a record of medications 
administered during the patient’s hospital stay. 

The Ministry provided eCHN with about 
$28.5 million in funding from 1998/99 through 
2005/06. Under its current agreement with the 
Hospital for Sick Children, the Ministry is providing 
eCHN with an additional $8 million annually for 
operational expenses and $200,000 for each new 
hospital or site connected to it. As of March 31, 
2009, 100 hospitals had been connected. 

In 2007, SSHA conducted a study to explore 
the feasibility of expanding the use of eCHN to 
include the adult population and develop it into the 
“first-generation” EHR. Although this is still under 
consideration, the 2009 Strategic Plan is unclear on 
how eCHN will fit into the EHR initiative. 


The Diabetes Registry 
The Diabetes Registry is projected to cost $54 mil- 
lion. The government’s original target date for 
the first release of the Diabetes Registry was April 
2009, as indicated in a July 12, 2008, press release 
announcing its launch: 


The strategy includes an on-line regis- 

try that will enable self-care by giving 
patients access to information and educa- 
tional tools that empower them to manage 
their disease. The registry will also give 
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health-care providers the ability to easily 
check patient records, access diagnostic 
information, and send patient alerts. The 
registry is set to come on-line starting 
spring 2009. 


However, this release has not occurred, and 
the proposed timing of this release of the Diabetes 
Registry has shifted regularly and significantly. For 
example, the following documents have provided 
the following dates for the proposed first release: 
@ in the August 2008 Report to Management 
Board of Cabinet—April 2009; 

e inthe November 2008 project charter—June 
2009; 

@ inthe March 2009 eHealth Strategic Plan— 
April 2010; and 

@ inthe June 2009 eHealth Ontario “rebaseline 
exercise” —fall 2010. 

The expected content of the release has also 
diminished over time. Original plans to have the 
first release of the registry available to all Ontarians 
have been adjusted to a “limited production rollout” 
to three sites within each of two LHINs. In short, 
deliverables have been significantly reduced since 
the government first publicly committed to the 
registry. 

One of the first activities undertaken under the 
Diabetes Registry initiative has been the gathering 
of diabetes patient and provider data. This is being 
done under a sub-project known as the Baseline 
Diabetes Dataset Initiative (BDDI). Data are being 
collected under the BDDI manually, at a particular 
point in time. Unfortunately, there is no mechan- 
ism currently in place for updating the information 
gathered. It is assumed under this initiative that all 
providers across Ontario will voluntarily provide 
their patients’ diabetes-related information in a 
timely manner and in the form required to populate 
the registry—something that remains to be seen. 
Also, it is not clear how and when the registry will 
be populated with the data manually gathered dur- 
ing the BDDI. 
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The Ontario College of Family Physicians 
expressed concerns about the Diabetes Registry 
in its Discussion Document: eHealth and Family 
Medicine. It noted that if the registry makes patient 
management more complex by requiring data to be 
entered twice (into the registry system as well as 
into the physician’s medical record system, whether 
that be paper-based or electronic), “the uptake, 
regardless of incentives, will be limited.” 

The Diabetes Registry project currently excludes 
children and uninsured individuals from its scope. 
Children have been excluded because of unresolved 
consent-management issues, and we saw no plans 
for eventually including them in future. Uninsured 
individuals were being excluded because OHIP’s 
claims information database is being used as the 
key source for the patient population. No provisions 
have yet been made to include in the registry, and 
ultimately in the EHR initiative, those individuals 
who are not covered by OHIP, such as those Ontar- 
ians who receive health-care coverage directly from 
the federal government (war veterans, inmates in 
federal corrections facilities, and members of First 
Nations, the RCMP, and the Canadian forces). 

The agency has decided to involve Infrastructure 
Ontario in the procurement process and thereby 
attempt to finance the project through the govern- 
ment’s Alternative Funding Procurement Model, 
which typically involves partnering and sharing 
financial risks with the private sector to the greatest 
extent possible. As a final point, at the time of our 
audit, the planned ratio of consultants to in-house 
staff working on the project of 75:25 has not been 
met. Reliance on consultants continued to be 
heavier than planned. 


Integrating the Systems and Linking to a Portal 
The system that will integrate the aforementioned 
applications and information systems and link 
them to a portal or viewer is known as the “Health 
Information Access Layer” (HIAL). Development of 
Ontario’s HIAL is currently at the infancy stage. 


In the absence of such a system, all of the vari- 
ous applications have been operating independ- 
ently. One consequence of this is that each project 
team was incorporating its own privacy and con- 
sent controls into its systems, and these efforts were 
not integrated. 

A $237-million integration-services project to 
provide the foundation for the HIAL by ensuring 
that all EHR-related systems can share information 
had yet to start at the time of our audit. A consent- 
management module to allow patients to partially 
control how their health information is shared 
amongst the different systems had also yet to be 
developed. The request for quotations for building 
the backbone of this integration system had been 
rescheduled from January to June 2009, owing to 
the transition from SSHA to the eHealth Ontario 
agency. 

The eHealth portal project is aimed at 
developing the on-line interface that will ultim- 
ately be the public window to EHR services. The 
project includes the establishment of standards 
and processes that will allow for an integrated 
view of clinical and personal health information. 
For example, the portal is being designed to make 
it possible for both clinicians and their patients to 
simultaneously access—from anywhere and at any 
time—laboratory test results from the Laboratory 
Information System and drug profiles from the 
Drug Information System. 

The original plan was for the portal to be fully 
available by June 2009. This did not occur, and the 
current plan has revised this as follows: 

e The consumer portal, to be focused on provid- 
ing patients and their families with access to 
their personal health data and the ability to 
self-manage certain aspects of their health 
care, is now projected to be available some- 
time in early 2011 and as a limited production 
roll-out rather than the full roll-out originally 
contemplated. Achieving this will require the 
integration of registration and log-on services, 
laboratory test results, the patient’s drug his- 
tory, the Diabetes Registry, health-knowledge 


resources, diagnostic imaging, an application 
to deal with immunizations and vaccine 
orders and distribution, and an application 
for managing communicable-disease out- 
breaks. The next step in this complex process 
is completing a consumer strategy, slated for 
December 2009. 

e The clinician portal, to be focused on provid- 
ing clinicians with tools, knowledge, and 
access to a comprehensive medical history of 
their patients, is now to be rolled out to only 
250 clinicians initially in spring 2010. With 
this roll-out, the agency plans to provide a 
more limited range of data than called for in 
the original plan. The integration of the portal 
with the Diabetes Registry is now planned to 
take place in the 2011/12 fiscal year. 


Implementation and Adoption Program 
The objective of this program is to support the suc- 
cessful implementation and adoption by the user 
community for each of the specific EHR applica- 
tions. Its cost over the four-year period to 2012 is 
estimated to be $340.6 million. 

Originally, $120 million was budgeted to be 
spent in the 2009/10 fiscal year to facilitate the 
April 2009 roll-out of the Diabetes Registry, the 
Client Registry, the Provider Registry, and the 
portal to all health-care providers in the diabetes 
circle of care (that is, the primary-care physicians 
and specialists involved in diabetes diagnosis, treat- 
ment, education, and related services) in Ontario’s 
14 LHINs. This roll-out was also to include consent- 
ing patients with diabetes across the province. 
However, since none of these applications are as 
yet completed, none of these implementation and 
adoption plans were under way at the time of our 
audit. 

In summary, Ontario’s EHR projects are 
behind schedule and struggling to deliver on their 
mandates. As well, integrating them so that they 
work together to collectively deliver an EHR to the 
medical community and all Ontarians remains a 
challenge. 
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THE USE AND PROCUREMENT 
OF CONSULTANTS 


Typically, consultants are considerably more expen- 
sive than full-time employees—a consultant with 

a rate of $300 per hour would collect more than 
$500,000 over a full year. Even the most experi- 
enced full-time employees are paid considerably 
less. Consultants also often carry their knowledge 
out the door with them when their contract termin- 
ates. Ensuring a full and appropriate knowledge 
transfer from consultants to employees, to enable 
the employees to operate or continue to develop 
the systems the consultants worked on without 

the need for outside help, is often easier said than 
done. 

The primary role of consultants should be to 
supplement internal resources and provide special- 
ist expertise and advice, usually for short-term per- 
iods. Using a consulting firm to build and deliver a 
complete, fully functional system is also an option. 
It may in fact appear to be a preferred option in 
times of hiring freezes, where ministries find it 
challenging to acquire the resources they need to 
deal with new strategic initiatives such as eHealth. 
But this approach has its risks. When it is used, 
organizations need clear policies and processes for 
the use of vendors of record and for the competi- 
tive selection of consultants. Such policies should 
require that assignments and their deliverables be 
well defined up front and that consultants transfer 
their knowledge to internal staff. This reduces the 
organization’s future dependency on consultants 
and the risk that consultants will hold the organiza- 
tion to ransom by threatening to walk out the door. 
In addition, payment arrangements are best based 
on achieved results rather than billable hours. 

It is also critical for an organization to have suf- 
ficient technical knowledge in house to effectively 
oversee and manage the work of its consultants. In 
previous audits of major government IT projects, 
we have noted that this was often lacking. We have 
also noted that a tendency to acquire resources in 
reaction to immediate rather than long-term needs 
often contributes to an overreliance on consultants. 
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Extent of Reliance on Consultants 


The Ontario government launched its EHR initia- 
tive in the early 2000s and the eHealth Program 
Branch was formed in early 2005. Yet we noted that 
even in 2009 the reliance on consultants at the Min- 
istry and the eHealth Ontario agency was, and con- 
tinues to be, very heavy. For example, at the time of 
its amalgamation into the eHealth Ontario agency 
in April 2009, the Ministry had more than 300 EHR 
consultants on contract, as compared to 27 eHealth 
employees. In a 2002 audit of government-wide 
consulting practices we were also critical of SSHA’s 
reliance on consultants. Since that time, SSHA has 
improved in this regard and as of September 30, 
2008, had approximately 560 full-time employees 
and 110 consultants. However, as shown in Fig- 

ure 4, there has been a dramatic increase in the 
number and ceiling prices of consulting contracts 
awarded by the Ministry’s eHealth Program Branch 
since the 2002/03 fiscal year, reflecting the Min- 
istry’s recent expanded efforts on EHR projects. 

We believe overseeing and monitoring consult- 
ant performance should be the responsibility of 
government employees. However, in a number of 
cases, consultants have been planning, supervising, 
or managing the work of other consultants. Several 
ministry senior management positions were held by 
consultants. Over 40% of staff at eHealth Ontario 
were consultants, and consultants filled 25% of 
senior management staff positions. On a combined 
basis, many EHR consultants have been engaged 
for a number of years. We noted that a total of 
$11.3 million has been paid for eight consultants 
who have been engaged on EHR-related projects 
dating as far back as 2001. 

This use of consultants to fill senior management 
positions continued even after the eHealth Ontario 
agency was formed. During the eHealth Ontario 
agency’s transition period, the CEO appointed four 
new senior vice presidents to report directly to her, 
and all four of these appointees were consultants. 
Of particular concern was that one of these con- 
sultants was a senior partner with the consulting 





Figure 4: Increase in Consulting Contracts at the 
Ministry’s eHealth Program Branch, 
2002/03-January 23, 2009 


Source of data: Ministry of Health and Long-Term Care 


Cumulative 

# of Consulting Contract Ceiling 

Contracts Price of New 

Fiscal Year Awarded Contracts ($ 000) 

2002/03 1 375 
(2003/04 8G 

2004/05 24 4,872 

2005/06 18 2,825 

2006/07 31 4,772 

2007/08 81 10,160 

2008/09* 328 38,252 





* to January 23, 2009 


firm that had been and continued thereafter to be 
the beneficiary of significant consulting contracts. 
Specifically, this firm had already been hired by the 
Ministry to develop the eHealth Strategy. It had fur- 
ther been hired to lead the eHealth Ontario transi- 
tion process and had also been awarded significant 
contracts for work on the Diabetes Registry project, 
the Drug Information System project, and the 
Client, Provider, and patient-consent registries. We 
questioned this particular appointment because, at 
a minimum, it created a perceived conflict of inter- 
est between this individual’s role in representing 
and protecting the province’s interests as a senior 
vice president of a Crown agency and the individ- 
ual’s role in furthering the interests of the consult- 
ing firm he worked for. 

We were also concerned about the extent to 
which consultants have been engaged in adminis- 
trative activities, many of which we believe could 
have more economically performed by salaried 
staff. Consultants have been paid, at high per diem 
rates, to prepare, review, and/or edit voice-mail 
greetings, thank-you letters, internal memos, 
Intranet pages, seasonal party communications, 
and documents prepared by other consultants. 


Professional editing services can be acquired ona 
contractual basis for rates ranging from $50 to $65 
per hour. The eHealth Ontario agency was paying 
rates in the $300-per-hour range for these services. 

We believe the main reason that the Ministry 
overused consultants was its desire to immediately 
respond to project work needs. We saw little evi- 
dence of well-thought-out long-term planning, such 
as formal forecasting of the overall EHR resources 
needed—or even the resources needed on a project- 
by-project basis—so that the most economical 
procurement arrangements could be planned for 
and made in advance. There was no overall plan 
that combined all of the known future resource 
requirements over the full time frame of the EHR 
initiative, despite the initiatives being identified 
as a government priority as far back as 2000. The 
Ministry continuously engaged consultants to lead 
its various EHR project teams, with little effort 
made to hire internal resources or develop in-house 
capabilities for managing or executing the projects. 
In-house staff were regularly recruited only for 
work relating to administration, policy-making, 
or finance. And even in those areas, we noted that 
consultants prepared ministry Cabinet submissions 
and the budgets necessary to develop the Ministry’s 
results-based plans. 

We also looked at whether attempts were 
made to ensure that knowledge was transferred to 
in-house staff to reduce the future need for consult- 
ants. For 40 of 45 consulting contracts sampled at 
the Ministry, there was no attempt upon contract 
completion to assess or evaluate what, if any, know- 
ledge transfer had occurred. We noted that ministry 
documentation repeatedly used the rationale of 
“urgency” or “shortage of internal resources with 
the required skills and knowledge” as a justification 
for renewing its consulting contracts. 


Procurement at the eHealth Ontario Agency 


The focus of much media attention and discussion 
at the Legislature has been the procurement of 
eHealth consultants between October 2008 and 
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June 2009. This was the transition period when 
SSHA’s operations and the Ministry’s eHealth 
Program Branch were being amalgamated into 

the new eHealth Ontario agency. Excluding the 
value of contract extensions when consultants 
under contract with the Ministry’s eHealth Pro- 
gram Branch were taken over by eHealth Ontario, 
eHealth Ontario awarded $5.6 million in consulting 
contracts during this nine-month period. None of 
these consultants were competitively procured, 

nor were there documented rationales prepared 

to justify sole-sourcing the work. The agency CEO 
arranged for 95% of it, with no procurement or 
human resources staff at the Ministry or the agency 
being involved. The CEO basically overrode all of 
the normal procedures and controls used by SSHA’s 
procurement department. Most of the contracts 
were signed only after the work had actually begun. 
One of the most questionable of these contracts was 
with a recruiting firm hired for $1 million to assist 
in the filling of 15 senior management positions. 
This work could clearly have been put out to tender 
or, at the very least, given to several firms who spe- 
cialized in recruiting for different functional areas. 
As well, a typical practice for such contracts is to 
pay a part of the fee up front, part upon the short- 
listing of appropriate candidates, and part upon the 
client’s hiring the successful candidate. However, 
this firm was paid most of the full fee up front. We 
noted that only five of the 15 positions were filled 
when the contract was terminated, yet no money 
was requested by, or returned to, the agency. 

The eHealth Ontario agency’s procurement 
policy did require that the CEO “ensure that 
there are adequate processes relating to procure- 
ment and, in particular, to non-competitive 
procurement.” The circumstances allowing for 
non-competitive procurement were, quite rightly 
in our view, very limited. They included “an 
unforeseeable situation of urgency or emergency,” 
situations where “a competitive procurement pro- 
cess could reasonably be expected to compromise 
eHealth Ontario’s confidentiality, cause economic 
disruption, or otherwise be contrary to the public 
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interest,” situations where it is necessary “to rec- 
ognize exclusive rights, such as exclusive licenses, 
copyright and patent rights,” and situations where 
“the goods or services can be supplied only by a 
particular vendor and no alternative or substitute 
exists.” However, the policy did not require that the 
rationale behind any decisions to procure services 
non-competitively be documented. The policy 
also did not require that management decisions 

to forego competitive bidding, even for significant 
contracts, be disclosed to the board or to a com- 
mittee of the board. Accordingly, the rationale for 
these decisions was either poorly documented or 
not documented at all, and the CEO applied these 
exemptions very liberally. 

During the months that the CEO was author- 
izing these procurements, the board was provided 
with very little information about them. As 
indicated by eHealth Ontario’s board minutes, on 
November 26, 2008, the CEO advised the board that 
“procurement of transition-team resources is being 
managed according to legacy (SSHA) organiza- 
tional policies and Management Board of Cabinet 
principles.” We found this to have been somewhat 
misleading, since the board was unaware that it 
was the exemptions to these policies that were 
being applied as opposed to the policies’ regular 
competitive procurement provisions. In this regard, 
we noted that at the same meeting, the board 
approved a guideline for the delegation of signing 
and spending authority, requiring that all com- 
mitments and payments with a value greater than 
$1.5 million be approved by the board. Commit- 
ments and payments for lesser amounts could be 
approved by the CEO alone. However, the guideline 
also stated that: 


The CEO shall report to the board in writ- 
ing all commitments and authorization of 
payments since the last report to the board 
with a value exceeding $1,000,000 as well 
as any commitments made and authoriza- 
tion of payments approved of particular 
importance regardless of value that: 


1. present unusually high business, repu- 
tational, or legal risks; 

2. could bring the Agency under public 
scrutiny; 

3. could affect the reputation of the 
Agency; 

4. involve controversial matters; or 

5. involve matters that are not within 
the ordinary course of business of the 
Agency. 


Despite this clause, the CEO brought none of the 
sole-sourced contracts that she personally arranged 
and awarded to the board’s attention. 

We met with the full board and asked about the 
extent to which it had been apprised of how these 
procurements of consultants were being handled. 
Members of the board advised us that they assumed 
procurements were being made through standard 
competitive processes until they became aware of 
sole-sourcing issues raised in the media. 

Although the board was unaware that these 
transition-related procurements were being sole- 
sourced, it was aware of a number of ministry 
consulting contracts being extended without com- 
petition upon their transfer to eHealth Ontario. The 
agency's board was somewhat reluctant to accept 
these contract extensions, as evidenced in the 
board minutes for April 8, 2009, which noted: 


a six-month contract extension was 
provided to consultants who were on 
contract with the Ministry of Health and 
Long-Term Care in its eHealth Program. 
The extension was important to allow for 
business continuity by enabling eHealth 
Ontario to retain resources that were 
deployed in “in flight” projects....The 
total value of the extended contracts 

was $36.6 million....The Directors 
expressed concern over the quantum of 
the extended contracts inherited from the 
Ministry. 


Aside from the aforementioned transition- 
period transactions, one of eHealth Ontario’s 
most questionable contracts involved a request for 
resources issued for privacy work. Not only did the 
request not clearly disclose the evaluation criteria 
and their respective weighting, but one of the firms 
bidding was aware of additional pertinent informa- 
tion that had not been disclosed to the other vend- 
ors. This firm had already been paid $25,700 under 
an earlier sole-sourced engagement with eHealth 
Ontario. This previous work was intended to be 
related to communications, but we noted that the 
actual work done was entirely privacy-related. 

This firm was selected as the successful bidder 
even though its bid price of $3.1 million was more 
than 500% higher than another qualified bidder’s 
price of $570,000. The bid price was also much 
higher than the approved budget for the project of 
$700,000. The eHealth Ontario agency asked only 
the selected firm to submit a second bid more in 
line with the approved budget. The firm sent in a 
revised proposal, reducing its total proposed price 
to $737,000, reducing the time frame over which 
the work would be performed from April-December 
2009 to April-June 2009, and significantly reducing 
the resources it would devote to the two main 
deliverables originally requested in the tender 
document. We found the scope of one of the main 
deliverables to be so significantly reduced in the 
revised proposal that, in our view, it was no longer 
possible to fairly evaluate it against the original 
request for resources issued or the proposals 
submitted by other bidders. However, no such 
evaluation was in fact attempted, nor was there any 
communication of the reduced scope of the work 
to the other bidders to give them an opportunity 
to likewise revise their submissions. We therefore 
found the competition process to be unfair. 

Further, the firm’s original proposal was to 
employ eight full-time resources (essentially all of 
its available staff) at rates per person ranging from 
$2,360 per day for three senior staff, $2,000 per 
day for three intermediate staff, and $1,600 per day 
for two junior staff. The revised proposal removed 
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three of these personnel. However, at almost the 
same time that it submitted the revised proposal, 
the firm emailed eHealth Ontario an unsolicited 
proposal that would engage the three consultants 
removed from the original proposal. The next 
day, eHealth Ontario prepared an exception-to- 
competitive-procurement request, and these three 
consultants were procured on a sole-source basis 
for $594,000 to work on the additional work pro- 
posed in the email, with the work to commence on 
the same date as the work under the first contract. 

The eHealth Ontario procurement policy clearly 
addressed unsolicited proposals, stating that they 
are “generally an attempt to obtain a sole source 
contract from the government and must adhere to 
the restrictions set out in Section 4.2.7 [the sec- 
tion that sets out the circumstances under which 
non-competitive procurement is allowed—such as 
emergency or urgent situations].” The unsolicited 
proposal for privacy-related services, in our view, 
fit none of the section’s special circumstances and 
should not have been simply accepted. 

There was absolutely no segregation of duties 
in the process of awarding this work. The same 
eHealth Ontario executive, who was one of the 
consultants appointed to be a senior vice president 
by the CEO, was responsible for the preparation 
of the business case, the request for resources, the 
evaluation of the proposals, and the exception-to- 
competitive-procurement request. 

Another issue that concerned us with respect 
to procurement at eHealth Ontario involved the 
arrangements for two of the consultants whom the 
CEO personally hired to serve as senior vice pres- 
idents. The CEO set their compensation levels. Each 
billed $2,700 daily for their services, plus charged 
for regular flights from Alberta (where they lived), 
for accommodation in Toronto, and for other 
expenses such as meals and incidentals. These 
two consultants were effectively paid at a rate of 
more than $700,000 annually before expenses for 
their services until they were abruptly terminated 
when their compensation arrangements became 
known to the media. One of them, who was also 
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paid a total of $78,500 for personal expenses, was 
afforded especially favourable treatment, in that his 
contract overrode the agency’s Travel and Expenses 
Policy for employees. This can best be illustrated 

by comparing his arrangements with those of the 
other Alberta consultant. eHealth Ontario paid for 
a “deluxe” one-bedroom apartment for this consult- 
ant while the other consultant stayed in a standard 
one-bedroom apartment. This consultant’s daily 
allowance for living expenses was $75 while the 
other consultant received $50 (agency policy for 
employees is a $45 per diem). The first consultant 
was also paid $1,600 for business-related meals 
before repaying this amount after a new Travel, 
Meal, and Hospitality Policy prohibiting consultants 
from being reimbursed for meals was approved in 
May 2009. He was further reimbursed for three 
conference fees, totalling $8,600, and incurred 
excessive flight expenses. We estimate that if this 
consultant had travelled as economically as the 
other consultant, the travel expenses would have 
been about 20% less, saving the agency $15,500. 
Both consultants were also reimbursed for a total 
of $7,300 in personal liability insurance and were 
granted significant corporate signing authority. 


Procurement at the Ministry's eHealth 
Program Branch 


The Ministry was required to follow the govern- 
ment’s established competitive procurement 
policies. We noted that the Ministry’s process for 
procuring consultants was often more form than 
substance or was otherwise questionable. For 
instance: 

e Vendor proposals were not properly evaluated 
for a significant number of the contracts we 
sampled, and most were approved without 
sign-off by all appropriate parties. We noted 
that a single ministry manager chose which 
vendors to invite and made the sole deci- 
sion on whom to hire for more than 30% of 
contracts sampled. Further, some contracts 
were signed on the same date as the closing 


date of the request for services. This again 
suggests that the competition was more pro- 
cedural than a true search for the best service 
provider. 

Seven of a sample of 11 IT architecture work 
contracts, worth almost $2 million, were 
awarded to a firm associated with a consult- 
ant who had been working with either the 
Ministry or SSHA since February 2000. This 
consultant was involved in five of these 

seven procurements by either approving the 
underlying business case or submitting the 
resource request form. The total value of these 
five awarded contracts was $1.3 million. The 
individual who signed off on one contract, for 
$687,000, did not have the signing author- 

ity for either the contract or the business 
case. This was detected only when ministry 
staff attempted to enter the contract into the 
government’s central accounting system and 
the system rejected it because it was for more 
than the individual’s signing-authority limit. 
Another of the contracts was awarded after an 
evaluation that included another consultant 
from the selected firm as one of the evalu- 
ators. This entire process had the appearance 
of a conflict of interest, and we were surprised 
that it was allowed to occur. 

The vendor evaluation for six of 12 highly 
technical IT architecture contracts sampled 
was performed solely by an administration 
manager instead of technically qualified staff. 
We noted contracts where the ministry staff 
who signed them were not properly author- 
ized to do so and where the authorizations 
were not properly documented. One individ- 
ual signed 12 contracts totalling $2.6 million 
that were above his signing-authority limit of 
$100,000. As with the example noted earlier, 
this was detected only when the contracts 
were entered into the government’s central 
accounting system, which rejected them 
because their amounts exceeeded the individ- 
ual’s signing-authority limit. 


e Some consultants claimed and were paid for 
hours worked on statutory holidays, without 
documented prior approval as required by 
policy. We noted the Ministry’s eHealth 
Program Branch office was closed and it was 
not the norm for these consultants to work 
overtime hours from home. 

We also had concerns about a Management 
Board exemption from competitive procurement 
that allowed one consulting firm to get a “foot in 
the door,” which in turn led to its receiving addi- 
tional contracts. Although this firm was undoubt- 
edly well qualified to assist the Ministry on eHealth 
matters, the inside position that this firm attained 
at the Ministry and, later, at eHealth Ontario gave 
it an advantage over other firms vying for EHR 
business. For example, this firm helped to develop 
the eHealth Strategy and led the eHealth Ontario 
transition process, and a senior partner with this 
firm acted for a time as a senior vice president of 
strategy and portfolio management at eHealth 
Ontario. The details of these arrangements are as 
follows. 

In August 2008, Management Board approved 
an exemption that allowed for a three-month 
waiver from competitive tendering for the acquisi- 
tion of $4.9 million worth of consulting services for 
contracts relating to the Diabetes Registry, the Drug 
Information System, the Client Registry, the regis- 
try of patient-consent data, and the work needed to 
integrate the applications and information systems 
and link them to a portal. The consulting firm in 
question was initially awarded almost $1.8 million 
in contracts under this exemption—specifically, 

a $530,000 contract for work relating to the 

Drug Information System, two contracts totalling 
$474,000 for work relating to the Diabetes Registry, 
and a $789,000 contract for work relating to the 
Client, Provider, and patient-consent registries. 

We noted that this firm was subsequently 
awarded, through the Ministry’s vendor-of-record 
process, the bulk of the ongoing work for both the 
Diabetes Registry and Drug Information System 
projects as they moved from one phase to another. 
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The Ministry parsed this work into multiple small 
contracts of less than $300,000, thereby bypass- 
ing certain ministry procurement requirements 
(contracts for more than $300,000 required review 
by the Supplies and Services Branch and, until 
July 1, 2008, sign-off at the assistant deputy min- 
ister level, and contracts for more than $750,000 
had to be tendered for open competition rather 
than be awarded via the Ministry’s vendor-of- 
record process). Over a 24-month period, the firm 
was awarded 10 of 13 Drug Information System 
contracts—with a value of $4.1 million out of a 
total $4.3 million worth of work—and, over an 
eight-month period, eight of 11 Diabetes Registry 
contracts—with a value of $2.9 million out of a 
total $3.2 million worth of work. If all of the work 
on each project had been combined, the Ministry 
would have had to use an open tendering process. 

In addition to our concerns about the practice 

of splitting work on each project into a number 

of smaller lower-value contracts, we had other 
concerns with the Ministry’s procurement practices 
relating to this firm: 

e The requests for services for two of the Dia- 
betes Registry contracts did not adequately 
describe the scope of the proposed work. 

As a result, the bid prices received varied 
widely—between $60,000 and $600,000 on 
one contract. The consulting firm in question 
submitted the highest bid and was awarded 
the contract. The Ministry had minimal docu- 
mentation to justify its awarding the contract 
to the highest bidder. 

e For five of the awarded contracts, the Min- 
istry was billed and paid $490,000 for work 
done prior to the contract start dates, with 
work commencing as far back as two months 
before the contract start dates. Work being 
performed prior to contracts being awarded 
is an indication that a competition is more 
of a formality than a true search for the best 
service provider. 

e We noted for the Drug information System 
contracts that only one of the competitions 
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went beyond the first evaluation stage and the 
questions asked of bidders in this competition 
largely related to work done under a previous 
contract that had been awarded to the con- 
sulting firm. This put that firm at a consider- 
able advantage in the competitive process. 
We also noted that the price quoted was not 
considered as a factor in the final evaluation, 
even though this is required by Management 
Board directives and the requests for services 
scoring information provided to vendors 
indicated that the price would be considered. 
The successful bidder, which again was the 
consulting firm in question, initially quoted a 
total fee that was higher than both the com- 
peting bid and the amount that the approved 
business case said was needed to obtain the 
deliverables. The firm was awarded the con- 
tract regardless and subsequently reduced its 
fees to align with the business case. 

We also noted that the firm being awarded 
this work was at times in communication 
with ministry management about upcoming 
contracts and contract renewals before the 
requests for services were released to other 
service providers. 

When an organization becomes heavily reli- 
ant on a particular consulting firm, there is 
always the risk that the firm will be able to use 
its privileged position to increase its fees. We 
found this to be the case here. In September 
2008, new vendor-of-record arrangements 
were put in place, and we noted that the firm 
raised all of its vendor-of-record per-diem 
rates by between 34% and 87%, depending 
on the position. For example, one consultant’s 
per diem increased at that time from $1,523 
per day to $2,850 per day, with the justifica- 
tion being that the consultant was working on 
higher-valued services. It is extremely difficult 
for ministries or agencies to challenge fee 
increases by entrenched consultants given the 
threat that they might walk off the job and 
leave the project in limbo. 


To sum up, although the Ministry's eHealth 
Program Branch seldom sole-sourced, we noted far 
too many instances where the competitive process 
for procuring consultants was more form than 
substance. Also, recurring contract renewals were 
all too common. In these instances, the two most 
common justifications for renewing contracts with 
the incumbent consultants were: 

e@ the consultants were already familiar with 

eHealth operations; and 

e rehiring the consultants would mitigate the 

risk of project delays or diminished service 
levels. 

In most of these instances, we did not agree that 
these almost routine renewals met the intent of the 
government’s policies on the use of consultants. 
The end result of the repeated use of these justifica- 
tions was that qualified consultants who were not 
already working on the EHR initiative were unable 
to compete fairly for contracts. 


Procurement at SSHA 


SSHA also relied heavily on consultants, but much 
less so than the eHealth Ontario agency or the 
Ministry. In our view, SSHA was making the most 
concerted attempts of the three organizations to 
minimize its use of consultants and become an 
organization predominantly staffed by full-time 
employees. 

Nevertheless, we found that SSHA did not 
always follow proper competitive procedures 
when acquiring consultants. We reviewed a sam- 
ple of files having a value of $11 million, where 
$10.5 million was for consulting services and 
$500,000 was for goods and services. About 70% 
of these purchases—valued at $7.4 million—were 
sole-sourced. In our view, this non-competitive 
procurement was adequately justified for only 
three of these files, worth $2.7 million. The 
justifications provided for sole-sourcing the 
remaining $4.7 million of procurements did not 
meet the requirements of SSHA’s sole-source 
procurement policy. For about $1 million of these 


procurements, there was no documentation jus- 
tifying the sole-sourcing. For another $1 million, 
the only reason cited was “urgency,” and in those 
cases we did not accept that these “urgent” situa- 
tions could not have been foreseen (under SSHA’s 
policies, sole-sourcing was allowed for only those 
urgent situations that were unforeseeable). The 
reason for sole-sourcing the remaining $2.7 million 
was the consultant’s previous experience with and 
knowledge of the project. This was not a sufficient 
justification for non-competitive procurement 
under SSHA’s policies. 

We also noted that for over 20% of these 
procurements, there was no business case or any 
other evidence that SSHA had considered other 
options before deciding to hire consultants. And for 
25% of the procurements, neither the contract nor 
the business case (if there was one) had specified 
deliverables. For another 30% of the procurements 
(covering six files), the contract deliverables were 
too vague for consultants not already familiar with 
SSHA’s operations to knowledgeably bid on the 
work. 

Finally, we also found that consulting contracts 
were repeatedly extended or renewed. Almost 40% 
of the consultants contracted by SSHA that we 
reviewed had been with SSHA for a considerable 
length of time, with the average being two years. 
Our concerns included the following: 

e SSHA initially engaged one consultant for 

a $32,000 deliverable. It then continued to 
renew his contract over a two-year period, 
eventually paying the consultant over 
$500,000. 

e SSHA initially engaged one consulting firm on 

a contract valued at $86,000. This firm was 
awarded renewals and extensions worth more 
than $600,000 over the following year. It was 
eventually paid a total of $710,454 over the 
period from June 2006 through March 2007, 
with no documented justification for any of the 
additional work beyond the initial contract. 

e Some consultants appeared to, in effect, 

arrange their own contract renewals. One 
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consultant who started working in December 
2006 on a three-month contract, and who had 
her contract previously renewed several times, 
proposed her own deliverables for her October 
2007 contract renewal. These were accepted 
as written. Another consultant submitted 
suggested deliverables that were then copied 
into the ministry business case justifying his 
hiring. The process was repeated when his 
contract was renewed. 

e Purchase orders, which should be issued 
before a contract is finalized, were often 
approved after contract work had com- 
menced. We found this to be the case in 
over 30% of purchase orders reviewed. For 
example, one purchase order for $271,000 
was approved in April 2007 but all con- 
tract costs were incurred before March 31, 
2007. Another purchase order for $737,000 
covered contract costs for the period from 
April 2007 through September 2007 but 
was not approved until June 1, 2007. A third 
purchase order for $44,100, for contract 
work taking place from August 1, 2007 to 
October 4, 2007, was approved on the date 
the contract expired. A fourth purchase order 
was approved on November 21, 2008, for 
$107,000—but the work it covered began 
more than seven weeks earlier, on October 1, 
2008 (extending to March 31, 2009). We 
also noted invoices paid that were for higher 
amounts than the purchase order specified. 
In one case, the invoice amount exceeded the 
purchase order by $8,700. 

We further tested 280 invoices from 15 dif- 
ferent consultants associated with 64 purchase 
orders issued from April 2006 to March 2009, 
and we noted that only 16 of the purchase orders 
were original requests for consulting services, 
with the remainder being contract extensions. The 
extensions involved increases in both the scope of 
the work and the time and resources allotted to 
complete the work. In one case, SSHA had acknow- 
ledged as early as June 2006 that it was overrelying 
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on consultants for the work on a certain project 

and that it should try to complete the project with 
internal staff—yet it continued to extend consultant 
contracts until March 2008. 





RECOMMENDATION 3 


The eHealth Ontario agency and the Ministry 
of Health and Long-Term Care should develop a 
project-resourcing plan that ensures that, over 
time, internal staff with an adequate depth of 
technological expertise are hired to oversee 

the completion of the Electronic Health Record 

initiative. The plan should incorporate the fol- 

lowing principles: 

e Requests for proposals for consulting servi- 
ces should include clear project deliverables 
with expected time frames for their comple- 
tion so that all bidders have sufficient infor- 
mation to make informed proposals. 

e Formal information technology life-cycle 
development processes—including processes 
for identifying project risks and developing 
strategies to effectively mitigate those 
risks—should be in place. 

e@ There should be oversight of consultant 
billings and the billings should be evalu- 
ated against project progress. Consultant 
contracts should also be structured such 
that significant payments are contingent on 
satisfactory delivery and that consultants 
share the risk if commitments regarding 
deliverables are not met. 





The Ministry and the government acknowledge 


that sound controllership and accountability 
are critical to proper financial management and 
expenditure control. As a result of recent events, 
the government, the Ministry, and eHealth 
Ontario have implemented a number of policy 
and procedural changes to improve accountabil- 
ity and restore the public’s confidence. 


Beginning in 2007, the Ministry consolidated 
eHealth-related initiatives across the Ministry 
under the authority of a single assistant deputy 
minister. The Ministry acknowledges that fee- 
for-service consultants were engaged to support 
both the planning and implementation of these 
initiatives. Recently, the Ministry created an 
eHealth Liaison Branch to provide ongoing 
oversight of eHealth initiatives. This branch will 
be staffed by employees of the Ontario Public 
Service. eHealth Ontario is also in the process 
of reducing its reliance on consultants through a 
fee-for-service conversion process and a recruit- 
ment drive. 

The Ministry has already begun to address 
and correct all of its procurement practices: it 
has created the Financial Management Branch to 
provide oversight of consulting services, restruc- 
tured its Supply and Financial Services Branch to 
enhance oversight capacity for all procurement 
within the Ministry, and instituted mandatory 
procurement training for all management staff. 





The eHealth Ontario board’s approved annual 


business plan will serve as the foundation for all 
project resourcing. Included in the development 
of this document was a staffing review where 
the workforce was examined and positions that 
had previously been held by fee-for-service 
consultants were targeted for conversion to 
full-time employment status. This re-balancing 
of the workforce will provide eHealth Ontario 
with the appropriate depth of expertise in its 
workforce and ensure continuity of knowledge 
transfer in a complex technology environment. 
The board has also approved procurement 
policies that lay the foundation for improved 
articulation of expected consultant perform- 
ance and evaluation of consultant work. These 
expectations are now documented in contracts 
with consultants. Financial controls have also 
been put in place to ensure that payments are 


based on performance and the achievement of 
results by consultants as described in the con- 
tract and in compliance with eHealth Ontario’s 
procurement policies. 

In spring 2009, the board also approved the 
I & IT Gateway Policy, which provides a gating 
mechanism to ensure large, complex technology 
projects meet specific project objectives as they 
move through the appropriate phases of the 
project life cycle. 


EXPENDITURE MANAGEMENT 


Neither the Ministry's eHealth Program Branch, 
SSHA, nor the eHealth Ontario agency had properly 
functioning systems for approving and monitoring 
payments. For example, we found that over 20% 

of the invoices paid for transition expenditures by 
the eHealth Ontario agency from October 2008 

to June 2009 were not properly approved or were 
paid without proper documentation being obtained 
to ensure that contract deliverables had been 
completed. 

We also tested a sample of expense transactions 
at SSHA for the period from April 2006 through 
March 2009. We found that in almost 40% of the 
cases where an assessment of alternative vendors 
was warranted, the documentation available was 
insufficient to support the vendor selected. These 
contracts were worth $1.8 million. In addition, pro- 
curement records were poorly organized. Many of 
the business cases or other supporting documents 
could not be found. 

We noted similar results in our test of 280 invoi- 
ces paid by SSHA reported on earlier. Specifically: 

e The procurement records were poorly organ- 
ized. Staff could not locate 22 of the 280 
invoices. Consequently, we were unable to 
verify whether these payments had been prop- 
erly authorized or recorded. 

e The majority of the documents supporting 
invoice payments consisted of timesheets 
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with little explanation of what work had 

actually been done by the consultants. Most 

timesheets simply identified the role of the 
consultant in the project, making it difficult 
for management to determine whether the 
consultant met his or her targets and what 
work had actually been accomplished. 

We also noted billing rate increases and bil- 

ling errors as contracts were extended and 

renewed: 

e One consultant’s rates increased from 
$1,100 per day to $1,500 from one pur- 
chase order to the next with no reason 
provided. 

e One consultant had been charging and 
was being paid $1,500 per day since March 
2007. His contract was for $1,300 per 
day. SSHA staff noticed the overpayment 
in November 2007 and identified a total 
overpayment of $30,050. But management 
decided that no action should be taken and 
continued to pay the higher rate to the con- 
sultant until the contract ended in March 
2008. The total amount of the overpayment 
was $45,750. 

e Acontract with one consulting firm allowed 
it to charge $1,160 per day for one consult- 
ant and $1,060 per day for the others. How- 
ever, we noted that two consultants were 
charging and being paid the higher rate. 

e One consultant charged for the same day 
on two different invoices, and both invoices 
were paid. 

e Over a six-month period, one consultant 
was paid a total of $42,000 to be on call. 
The eHealth Ontario agency used what we 
believe to be a more appropriate formula 
for calculating the amount to be paid for 
such work, and over a comparable six- 
month period, payments to the same con- 
sultant for being on call dropped to $1,900. 

e As noted earlier, project expenditures and 
deliverables were tracked separately. The 
absence of a high-level process to facilitate 
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comparison of costs to work completed 
made it all the more essential that adequate 
oversight mechanisms be in place to ensure 
that consultants were completing the work 
they were being paid for. In far too many 
cases, the necessary review and approval 
controls were lacking. 

Other examples of bypassed controls and lack of 
appropriate expenditure management included the 
eHealth Ontario CEO regularly approving her own 
expense claims rather than submitting them to the 
Chair or the board. As well, $197,000 was paid to a 
consultant, at a rate of $1,710 per day, for providing 
“executive assistance” to the CEO. She was termin- 
ated only after this arrangement was reported in 
the media. 

We also sampled 20 jobs at the eHealth Ontario 
agency, comparing their rates of pay with those for 
similar jobs in the Ontario Public Service (OPS). 
For 19 of them, the maximum of the eHealth 
Ontario job’s salary range exceeded that of the 
comparable OPS job by an average of 31%. 

These and the examples provided earlier have 
damaged the reputation of the eHealth Ontario 
agency. The agency is now faced with regaining 
the support and trust of clients and other stake- 
holders. It must also repair the morale of staff, 
who have lived through extended periods of 
uncertainty during which the agency’s governance 
and senior management structure has frequently 
changed. Improved procedures and policies must 
be formulated and adhered to. A start was made 
in July 2009, with Management Board of Cabinet’s 
approval of a new Procurement Directive requiring, 
with few exceptions, that all consulting services, 
irrespective of value, be competitively procured, 
and an addendum to the Travel, Meal and Hospital- 
ity Expenses Directive that prohibits ministries and 
agencies from reimbursing consultants for meal, 
hospitality, and incidental expenses. As discussed 
earlier, eHealth Ontario management has also 
taken a number of steps to improve the agency’s 
practices in these and other areas. 


But most importantly, eHealth Ontario, with 
appropriate oversight and guidance from both the 
Ministry and what we believe to be a well-qualified 
board, must now work at overcoming the challenges 
of completing and integrating the many application 
and database projects for which it is responsible so 
it can achieve the goal of delivering an Electronic 
Health Record system to Ontarians by 2015. 


RECOMMENDATION 4 





The eHealth Ontario agency and the Ministry 
of Health and Long-Term Care should establish 
appropriate review and approval mechanisms to 
ensure that procurement policies are followed 
and that any exceptions to such policies are 
made for sound and defensible business reasons 
and approved at an appropriate level. Signifi- 
cant exceptions should require board approval 
and be reported to the Minister of Health and 
Long-Term Care. 





The Ministry and eHealth Ontario fully support 


the findings of the Auditor General’s report and 
have already made changes to policies and pro- 
cedures relating to procurement. 





In the past months, strengthened procurement, 


conflict-of-interest, delegation-of-authority, 
and whistleblower policies, among others, have 
been approved. In addition, a review of current 
contracts has been conducted and a calendar 

of future procurements is being developed to 
ensure that, prior to the expiration of current 
agreements, a competitive procurement process 
can be conducted. 

The Interim Chief Executive Officer will also 
develop a quarterly review process that will 
enable senior management to provide the board 
with a signed declaration that all policies relat- 
ing to procurement have been followed. 
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Appendix—Chronology of Events Relating to the 


Electronic Health Record Initiative 





April 1996 ¢ Ontario government forms a Health Services Restructuring Commission to provide a framework 
and make recommendations for restructuring Ontario’s public hospitals and health service delivery 
system 


¢ Commission identifies lack of integration, lack of health information, and difficulty in sharing health 
information as key problems 





June 1999 © Commission submits the Ontario Health Information Management Action Plan to Minister of Health 
and Long-Term Care 

¢ Action Plan recommends acceleration of information and technology investments so that health-care 
information can be better captured, shared, and analyzed 
Action Plan envisions “an integrated health information network with an electronic consumer record 
at its core” 
Action Plan also recommends creation of an independent, arm’s-length entity to provide strong 
central leadership, manage implementation of action plan, and allocate financial resources 


September 2000 e First Ministers agree “to work together to strengthen a Canada-wide health infrastructure to improve 
quality, access, and timeliness of health care for Canadians” 

First Ministers also commit to develop Electronic Health Records and the common standards 
necessary to ensure the future compatibility of all the jurisdictions’ health-information networks 


2001 


government of Canada creates and funds Canada Health Infoway as an independent, not-for-profit 
Shared Governance Corporation 


Infoway’s members are Canada’s 14 federal, provincial, and territorial deputy ministers of health 
Infoway’s mission is to accelerate the development of electronic health information systems (such 
as Electronic Health Records) on a pan-Canadian basis 

Infoway’s goal is that 50% of Canadians will have an Electronic Health Record by 2010 and 100% 
will have an Electronic Health Record by 2016 


2002 e Ontario government creates the Smart Systems for Health Agency (SSHA) by regulation under the 
Development Corporations Act 
@ SSHA begins operations in April 2003 with a mandate to support Ministry of Health and Long-Term 
Care programs; it begins to build a private network to connect Ontario’s medical community 


2004 © Ministry's eHealth Program Branch created to establish and maintain an eHealth strategy and 
oversee its delivery, including the development of EHR applications and databases 


2005 


SSHA’s mandate clarified by regulation to one of providing and operating secure infrastructure to 
enable secure transfer of personal health information among sector providers and hosting personal- 
health-information applications 





November 2006 e Deloitte Consulting completes critical operational review of SSHA, commissioned by the Ministry 
of Health and Long-Term Care and SSHA’s board of directors—by this time SSHA has some 1,400 
network circuits deployed, costing more than $500,000 monthly to maintain 


March 2007 e Infoway’s 2007/08 corporate business plan projects that, in terms of EHR progress, Ontario will be 
ahead of Northwest Territories, Yukon, and Nunavut but behind all provinces by March 31, 2008 


August 2007 Office of Auditor General of Ontario reviews SSHA's efforts to address certain of Deloitte's key 
recommendations and notes progress made in some areas while more work required in others 


e Auditor General’s review identifies lack of an overall government eHealth strategy and Ontario's slow 
progress in overall Electronic Health Records achievements as continuing issues 


e federal and provincial Auditor Generals discuss possibility of collaborative audit of electronic health 
records 
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September 2007 


June 2008 


@ Ministry and SSHA sign an Affirmation of their Memorandum of Understanding 

@ SSHA’s mandate is to provide “the secure, integrated, province-wide information technology 
infrastructure to allow electronic communication among Ontario's health service providers” 

@ SSHA strikes a client experience task force to improve services to and relationships with clients, 
which identifies SSHA’s poor understanding of its customers and poor performance issues as key 
problems 





August 2008 


@ Ministry makes submission to Management Board of Cabinet, seeking and obtaining approval for 
“the establishment of an Electronic Health Record of clinical information for every patient, and that 
could be controlled by the patient, by 2015” 





September 2008 


October 2008 


March 2009 


April 2009 


May/June 2009 


¢ federal and five provincial Auditor General offices agree to conduct a collaborative audit on 
Electronic Health Records initiatives across Canada, with each jurisdiction issuing its own report 

¢ federal Auditor General is to combine observations from these reports with its own work on Canada 
Health Infoway operations and issue a Canada-wide report in 2010 

e Ontario government creates the eHealth Ontario agency to combine the activities and responsibilities 
of SSHA and Ministry's eHealth Program Branch into one organization responsible “for all aspects of 
eHealth in Ontario, including creating an Electronic Health Record for all Ontarians” 

e Ontario government also forms the eHealth Ontario agency’s first board; no members from SSHA’s 
board are invited to join the eHealth Ontario agency’s board; Premier appoints board Chair 


e Premier supports Chair's request regarding appointment of CEO after meeting with Chair's 
recommended candidate 

e Order-in-council is issued making CEO’s appointment official 

e Canada Health Infoway’s 2009/10 corporate business plan shows Ontario still lagging behind all 
provinces (although ahead of Northwest Territories, Yukon, and Nunavut) in developing Electronic 
Health Records 

e Canada Health Infoway’s business plan indicates that 17% of Canadians have an Electronic Health 
Record available to their health-care provider 

e Ontario government's eHealth Strategic Plan published 

Strategic Plan describes activities to be undertaken from 2009 through 2012 and sets target of 
delivering a fully functional Electronic Health Record system by 2015 


¢ Ministry and eHealth Ontario sign a Memorandum of Understanding and a Transfer Payment 
Agreement setting out their respective accountability 


¢ significant controversy over possible sole-sourcing of consultants and related costs erupts in the 
Legislature and the media 


Gere emcee 


June 2009 


¢ eHealth Ontario agency’s CEO and board Chair resign 





@)ii(q=s0) md a(-7-\0 (el co) a @{-0(-t0-1)(0)  @lnie-lale) 


Box 105, 15th Floor 
20 Dundas Street West 


Toronto, Ontario 
NY fo ©) 74 G2 


avatar [llel|uel els Wer, 





ISBN 978-1-4435-1037-0 (Print) 
ISBN 978-1-4435-1038-7 (PDF) 


+ 
shes Te 


i 
Hye 
sone 


cet 
{s Ht wy 
Cera 


a 
HEARN 
ay 
ga 
SY 


AF 
i 








